U 9%eB@sdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZmZddlZddlmmZddlZddlZddlmZddlm Z m!Z!m"Z"m#Z#m$Z$ddl%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+m,Z,d d d d d ddddddddg Z-dddd Z.iZ/ej0e(j1dddd Z2ej0e(j1dddd Z3ej0e(j1ddd d Z4ej0e(j1ddd!d"d Z5e(j1d#d$ej6j7ddej8j9dd#dddd$d#f d%d&d'dddd(d(d)d*dd+d,d-d.d,d/d0d1dZ:e(j1d2d3Z;e(j1d4d5Ze(j1d;d<Z?e(j1d,d)d=d>d?d@Z@e(j1dAdBZAe(j1dCdDZBe(j1dEddFdGZCe(j1dHdIZDe(j1ddJdKZEe(j1dLdMZFe(j1dNdOZGe(j1dPdQZHddRdSdTdUdVZIe(j1dWdXZJe(j1dYdZZKejLejMejNejOejPejQejRejOiZSe(j1dd[dZTe(j1d\d]ZUe(j1d$ddej8j9d#d$d$ej6j7df d^dd_d`ZVe(j1ejWd#d$ej6j7ddej8j9dd$dddd#d#dfdadZXe(j1ej6j7dfdd*dbdcdddZYe(j1ded.dfdgdhdiZZe(j1djdkZ[e(j1dldmZ\e(j1d#d$ej6j7ddej8j9ddd#ddd$dd#d#d$d#fdndoZ]e(j1dpdqZ^e(j1drdsZ_e(j1dtduZ`e(j1dvdwdxdydzZae(j1dwd{d|d}Zbe(j1dwd~d=dddZce(j1dd=d)dddZde(j1dddddZee(j1dddddZfe(j1dvddddZge(j1ej8j9fd6dwdvddddddZhe(j1ddddZie(j1ddd=dddZje(j1dd=dddZke(j1ddZlddEdddZmdS)zFunctions to export models into the ONNX IR format. These models can be loaded with the ONNX library and then converted to models which run on other deep learning frameworks.  annotationsN) AnyCallablecast CollectionDictListMappingOptionalSequenceSetTupleTypeUnion)_C) _constants_exporter_stateserrorssymbolic_caffe2symbolic_helper)GLOBALS) _beartype diagnostics jit_utilsonnx_proto_utils registrationis_in_onnx_exportselect_model_mode_for_exportdisable_apex_o2_state_dict_hooksetup_onnx_loggingexporter_contextexportmodel_signaturewarn_on_static_input_changeunpack_quantized_tensorexport_to_pretty_stringunconvertible_opsregister_custom_op_symbolicunregister_custom_op_symbolicbool)returncCstjS)z3Returns whether it is in the middle of ONNX export.)rin_onnx_exportr-r-O/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/torch/onnx/utils.pyrFsz_C_onnx.TrainingMode)modec cst|tjs tdt|dd}t|dr|j}|tjjksP|tjjkrv|rvdt _ t j dkr|t dt j dndt _ |t _|tjjkr|dn|tjjkr|dz d VW5t|dr|tjjks||Xd S) a"A context manager to temporarily set the training mode of ``model`` to ``mode``, resetting it when we exit the with-block. Args: model: Same type and meaning as ``model`` arg to :func:`export`. mode: Same type and meaning as ``training`` arg to :func:`export`. z:'mode' should be a torch.onnx.TrainingMode enum, but got 'z'.FtrainingT zEYou are exporting the model in training mode with onnx opset version zn. Opset versions lower than opset 12 will not be able to export nodes such as Dropout and BatchNorm correctly.N) isinstance_C_onnx TrainingMode TypeErrortypehasattrr0ZTRAININGZPRESERVErZexport_trainingexport_onnx_opset_versionwarningswarnZ training_modetrainEVAL)modelr/Zoriginally_trainingr-r-r.rPs6           z0Union[torch.nn.Module, torch.jit.ScriptFunction]r=c cst|tjjsi}|D]d}|jD]2\}}t|jdkr(||krNi||<||||<q(||kr||D]}|j |qlqz dVW5|D]$\}}|D]\}}||j|<qqXnz dVW5XdS)NZO2StateDictHook) r2torchjitScriptFunctionmodulesZ_state_dict_hooksitemsr6__name__pop)r=Z model_hooksmodulekeyhookZm_mapr-r-r.rs&     verboseccs<tj}|s|rtjz dVW5|s6tjXdSN)r?onnxZis_onnx_log_enabledZ enable_logZ disable_log)rJZis_originally_enabledr-r-r.r s   )r/rJc csdt||P}t|<}t|(}t}||||fVW5QRXW5QRXW5QRXW5QRXdSrK)rrr rZ create_export_diagnostic_context)r=r/rJZmode_ctxZapex_ctxZlog_ctxZdiagnostic_ctxr-r-r.r!s&TFzHUnion[torch.nn.Module, torch.jit.ScriptModule, torch.jit.ScriptFunction]z$Union[Tuple[Any, ...], torch.Tensor]zUnion[str, io.BytesIO]zOptional[Sequence[str]]z_C_onnx.OperatorExportTypesz Optional[int]zMOptional[Union[Mapping[str, Mapping[int, str]], Mapping[str, Sequence[int]]]]zOptional[bool]zOptional[Mapping[str, int]]z.Union[bool, Collection[Type[torch.nn.Module]]]None)r=argsf export_paramsrJr0 input_names output_namesoperator_export_type opset_versiondo_constant_folding dynamic_axeskeep_initializers_as_inputs custom_opsetsexport_modules_as_functionsautograd_inliningr+cCs,t|||||||||| | | | | ||ddS)a9Exports a model into ONNX format. If ``model`` is not a :class:`torch.jit.ScriptModule` nor a :class:`torch.jit.ScriptFunction`, this runs ``model`` once in order to convert it to a TorchScript graph to be exported (the equivalent of :func:`torch.jit.trace`). Thus this has the same limited support for dynamic control flow as :func:`torch.jit.trace`. Args: model (:class:`torch.nn.Module`, :class:`torch.jit.ScriptModule` or :class:`torch.jit.ScriptFunction`): the model to be exported. args (tuple or torch.Tensor): args can be structured either as: 1. ONLY A TUPLE OF ARGUMENTS:: args = (x, y, z) The tuple should contain model inputs such that ``model(*args)`` is a valid invocation of the model. Any non-Tensor arguments will be hard-coded into the exported model; any Tensor arguments will become inputs of the exported model, in the order they occur in the tuple. 2. A TENSOR:: args = torch.Tensor([1]) This is equivalent to a 1-ary tuple of that Tensor. 3. A TUPLE OF ARGUMENTS ENDING WITH A DICTIONARY OF NAMED ARGUMENTS:: args = ( x, { "y": input_y, "z": input_z } ) All but the last element of the tuple will be passed as non-keyword arguments, and named arguments will be set from the last element. If a named argument is not present in the dictionary, it is assigned the default value, or None if a default value is not provided. .. note:: If a dictionary is the last element of the args tuple, it will be interpreted as containing named arguments. In order to pass a dict as the last non-keyword arg, provide an empty dict as the last element of the args tuple. For example, instead of:: torch.onnx.export( model, ( x, # WRONG: will be interpreted as named arguments {y: z} ), "test.onnx.pb" ) Write:: torch.onnx.export( model, ( x, {y: z}, {} ), "test.onnx.pb" ) f: a file-like object (such that ``f.fileno()`` returns a file descriptor) or a string containing a file name. A binary protocol buffer will be written to this file. export_params (bool, default True): if True, all parameters will be exported. Set this to False if you want to export an untrained model. In this case, the exported model will first take all of its parameters as arguments, with the ordering as specified by ``model.state_dict().values()`` verbose (bool, default False): if True, prints a description of the model being exported to stdout. In addition, the final ONNX graph will include the field ``doc_string``` from the exported model which mentions the source code locations for ``model``. If True, ONNX exporter logging will be turned on. training (enum, default TrainingMode.EVAL): * ``TrainingMode.EVAL``: export the model in inference mode. * ``TrainingMode.PRESERVE``: export the model in inference mode if model.training is False and in training mode if model.training is True. * ``TrainingMode.TRAINING``: export the model in training mode. Disables optimizations which might interfere with training. input_names (list of str, default empty list): names to assign to the input nodes of the graph, in order. output_names (list of str, default empty list): names to assign to the output nodes of the graph, in order. operator_export_type (enum, default OperatorExportTypes.ONNX): * ``OperatorExportTypes.ONNX``: Export all ops as regular ONNX ops (in the default opset domain). * ``OperatorExportTypes.ONNX_FALLTHROUGH``: Try to convert all ops to standard ONNX ops in the default opset domain. If unable to do so (e.g. because support has not been added to convert a particular torch op to ONNX), fall back to exporting the op into a custom opset domain without conversion. Applies to `custom ops `_ as well as ATen ops. For the exported model to be usable, the runtime must support these non-standard ops. * ``OperatorExportTypes.ONNX_ATEN``: All ATen ops (in the TorchScript namespace "aten") are exported as ATen ops (in opset domain "org.pytorch.aten"). `ATen `_ is PyTorch's built-in tensor library, so this instructs the runtime to use PyTorch's implementation of these ops. .. warning:: Models exported this way are probably runnable only by Caffe2. This may be useful if the numeric differences in implementations of operators are causing large differences in behavior between PyTorch and Caffe2 (which is more common on untrained models). * ``OperatorExportTypes.ONNX_ATEN_FALLBACK``: Try to export each ATen op (in the TorchScript namespace "aten") as a regular ONNX op. If we are unable to do so (e.g. because support has not been added to convert a particular torch op to ONNX), fall back to exporting an ATen op. See documentation on OperatorExportTypes.ONNX_ATEN for context. For example:: graph(%0 : Float): %3 : int = prim::Constant[value=0]() # conversion unsupported %4 : Float = aten::triu(%0, %3) # conversion supported %5 : Float = aten::mul(%4, %0) return (%5) Assuming ``aten::triu`` is not supported in ONNX, this will be exported as:: graph(%0 : Float): %1 : Long() = onnx::Constant[value={0}]() # not converted %2 : Float = aten::ATen[operator="triu"](%0, %1) # converted %3 : Float = onnx::Mul(%2, %0) return (%3) If PyTorch was built with Caffe2 (i.e. with ``BUILD_CAFFE2=1``), then Caffe2-specific behavior will be enabled, including special support for ops are produced by the modules described in `Quantization `_. .. warning:: Models exported this way are probably runnable only by Caffe2. opset_version (int, default 17): The version of the `default (ai.onnx) opset `_ to target. Must be >= 7 and <= 17. do_constant_folding (bool, default True): Apply the constant-folding optimization. Constant-folding will replace some of the ops that have all constant inputs with pre-computed constant nodes. dynamic_axes (dict[string, dict[int, string]] or dict[string, list(int)], default empty dict): By default the exported model will have the shapes of all input and output tensors set to exactly match those given in ``args``. To specify axes of tensors as dynamic (i.e. known only at run-time), set ``dynamic_axes`` to a dict with schema: * KEY (str): an input or output name. Each name must also be provided in ``input_names`` or ``output_names``. * VALUE (dict or list): If a dict, keys are axis indices and values are axis names. If a list, each element is an axis index. For example:: class SumModule(torch.nn.Module): def forward(self, x): return torch.sum(x, dim=1) torch.onnx.export( SumModule(), (torch.ones(2, 2),), "onnx.pb", input_names=["x"], output_names=["sum"] ) Produces:: input { name: "x" ... shape { dim { dim_value: 2 # axis 0 } dim { dim_value: 2 # axis 1 ... output { name: "sum" ... shape { dim { dim_value: 2 # axis 0 ... While:: torch.onnx.export( SumModule(), (torch.ones(2, 2),), "onnx.pb", input_names=["x"], output_names=["sum"], dynamic_axes={ # dict value: manually named axes "x": {0: "my_custom_axis_name"}, # list value: automatic names "sum": [0], } ) Produces:: input { name: "x" ... shape { dim { dim_param: "my_custom_axis_name" # axis 0 } dim { dim_value: 2 # axis 1 ... output { name: "sum" ... shape { dim { dim_param: "sum_dynamic_axes_1" # axis 0 ... keep_initializers_as_inputs (bool, default None): If True, all the initializers (typically corresponding to parameters) in the exported graph will also be added as inputs to the graph. If False, then initializers are not added as inputs to the graph, and only the non-parameter inputs are added as inputs. This may allow for better optimizations (e.g. constant folding) by backends/runtimes. If True, `deduplicate_initializers` pass will not be executed. This means initializers with duplicated values will not be deduplicated and will be treated as distinct inputs to the graph. This allows different input initializers to be supplied at the runtime following export. If ``opset_version < 9``, initializers MUST be part of graph inputs and this argument will be ignored and the behavior will be equivalent to setting this argument to True. If None, then the behavior is chosen automatically as follows: * If ``operator_export_type=OperatorExportTypes.ONNX``, the behavior is equivalent to setting this argument to False. * Else, the behavior is equivalent to setting this argument to True. custom_opsets (dict[str, int], default empty dict): A dict with schema: * KEY (str): opset domain name * VALUE (int): opset version If a custom opset is referenced by ``model`` but not mentioned in this dictionary, the opset version is set to 1. Only custom opset domain name and version should be indicated through this argument. export_modules_as_functions (bool or set of type of nn.Module, default False): Flag to enable exporting all ``nn.Module`` forward calls as local functions in ONNX. Or a set to indicate the particular types of modules to export as local functions in ONNX. This feature requires ``opset_version`` >= 15, otherwise the export will fail. This is because ``opset_version`` < 15 implies IR version < 8, which means no local function support. Module variables will be exported as function attributes. There are two categories of function attributes. 1. Annotated attributes: class variables that have type annotations via `PEP 526-style `_ will be exported as attributes. Annotated attributes are not used inside the subgraph of ONNX local function because they are not created by PyTorch JIT tracing, but they may be used by consumers to determine whether or not to replace the function with a particular fused kernel. 2. Inferred attributes: variables that are used by operators inside the module. Attribute names will have prefix "inferred::". This is to differentiate from predefined attributes retrieved from python module annotations. Inferred attributes are used inside the subgraph of ONNX local function. * ``False`` (default): export ``nn.Module`` forward calls as fine grained nodes. * ``True``: export all ``nn.Module`` forward calls as local function nodes. * Set of type of nn.Module: export ``nn.Module`` forward calls as local function nodes, only if the type of the ``nn.Module`` is found in the set. autograd_inlining (bool, default True): Flag used to control whether to inline autograd functions. Refer to https://github.com/pytorch/pytorch/pull/74765 for more details. Raises: :class:`torch.onnx.errors.CheckerError`: If the ONNX checker detects an invalid ONNX graph. :class:`torch.onnx.errors.UnsupportedOperatorError`: If the ONNX graph cannot be exported because it uses an operator that is not supported by the exporter. :class:`torch.onnx.errors.OnnxExporterError`: Other errors that can occur during export. All errors are subclasses of :class:`errors.OnnxExporterError`. )rSrTrUrVrWrXrYrZN)_export)r=rNrOrPrJr0rQrRrSrTrUrVrWrXrYrZr-r-r.r"s(IcCsN|dkrdS|}|tjr0dS|ttjrJdSdS)Nzprim::ConstantFT) kindoutputr6Z isSubtypeOfrListType ofTensorsZ OptionalTypeZofTensor)node output_typer-r-r._is_constant_tensor_lists  rbcCs|D]}|D]}t||qt|rg}|D]4}||}||| || |q<| d| | tj}| |||qdS)Nzprim::ListConstruct)nodesblocks_split_tensor_list_constantsrbr]ZtoIValueZinsertConstantr`Z moveBeforeZ copyMetadataappendcreateZ insertBeforesetTyperr^r_ZreplaceAllUsesWith)gblockr`Zsubblockinputsvalinputlcr-r-r.re's$      rez_C.Graph)graphrS_disable_torch_constant_propfixed_batch_sizec Cs|dkr i}t|t|t|tjr:t|t||dkrVt|t ||t |t|t |rt |t |t|t|dt|t|t|dt|t||t|t|t|t|t|tjt||ttr\t||ttdtd|t|tjr|dkr|gn|}|dkrin|}t |||t |t!||}t |t|t"|dtj#t|t$|tj#|t|t%|t|t&|}t|tjrzt'||tj#Wn<t(k r~}zt)j*rn|j+ddkrnW5d}~XYnX|S)NFTz graph(%Pi): %Pq = quantized::nhwc2nchw(%Pi) %Pr = quantized::nchw2nhwc(%Pq) return (%Pr)z< graph(%Ri): return (%Ri)r>ScalarType UNKNOWN_SCALAR is an unexpected tensor scalar type!),r_jit_pass_inlineZ_jit_pass_inline_fork_waitZ_jit_pass_lintrrZZ(_jit_pass_onnx_autograd_function_processZ_jit_pass_lower_all_tuplesZ_jit_pass_constant_propagationreZ _jit_pass_dceZ _jit_pass_cse_jit_pass_onnx_lintZ&_jit_pass_canonicalize_graph_fuser_opsZ_jit_pass_peepholeZ_jit_pass_fuse_addmm*_jit_pass_onnx_remove_inplace_ops_for_onnxZ_jit_pass_onnx_preprocessZ#_jit_pass_prepare_division_for_onnxZ_jit_pass_onnx_remove_printZ _jit_pass_onnx_preprocess_caffe2rZ_quantized_opsclearZ'_jit_pass_onnx_unpack_quantized_weightsis_caffe2_aten_fallbackZ+_jit_pass_onnx_quantization_insert_permutesZ,_jit_pass_custom_pattern_based_rewrite_graphtextwrapdedent_jit_pass_erase_number_typesonnx_shape_inferenceZ&_jit_pass_onnx_set_dynamic_input_shapeZ_jit_pass_onnxZ#_jit_pass_onnx_scalar_type_analysisr8Z_jit_pass_onnx_peephole4_jit_pass_dce_allow_deleting_nodes_with_side_effectsZ_jit_pass_canonicalize)_jit_pass_onnx_graph_shape_type_inference RuntimeErrorr3_CAFFE2_ATEN_FALLBACKrN) rorSrprq params_dictrVrQrFexcr-r-r._optimize_graph>s                                       rcCsrt|d|dD]Z\}}t|trLt|t|krld}t|qt|tr||krd}t|qdS)aYWarns that changes to input dictionaries and strings won't take effect in the traced ONNX graph. We accept dictionaries and strings as ONNX inputs, but they should be only for configuration use. we detect here if these inputs are modified, and if so we warn the user that the changes won't take effect in the traced ONNX graph. raQWe detected that you are modifying a dictionary that is an input to your model. Note that dictionaries are allowed as inputs in ONNX but they should be handled with care. Usages of dictionaries is not recommended, and should not be used except for configuration use. Also note that the order and values of the keys must remain the same. zvThe model seems to have string inputs/outputs. Note that strings will not appear as inputs/outputs of the ONNX graph. N)zipr2dictlistkeysr9r:str)Z input_statesrmZ traced_inputwarningr-r-r.r$s   cCs8||jk r4tjr4|dkr0td|d|dd}|S)zVResolves the arguments that are ignored when export_type != operator_export_type.ONNX.T'zu' can be set to True only when 'operator_export_type' is `ONNX`. Since 'operator_export_type' is not set to 'ONNX', 'z' argument will be ignored.F)ONNXr3rr9r:)Zarg_name arg_valuerSr-r-r._resolve_args_by_export_typesrint)rWrSrTcCsJ|dkr|dkrtddS|dkr*dn|}|dkrF|tjjkrFd}|S)aeDecides whether the initializers in the graph should be listed as ONNX graph inputs. This method encapsulates the logic to decide whether the initializers in the graph should be listed as ONNX graph inputs (i.e., whether to choose ONNX IR v3 or v4). If keep_initializers_as_inputs is not specified (None), then we decide whether to keep initializers as graph inputs (val_keep_init_as_ip) based on export type. If export type is ONNX, then do not keep initializers as input (val_keep_init_as_ip=False). For all other export types keep initializers as input (val_keep_init_as_ip=True). If keep_initializers_as_inputs is specified, then respect it. Unless opset version <= 8, in which case it must be ignored because for opset version <= 8, all initializers MUST be part of graph input (only ONNX IR v3 is allowed), i.e. val_keep_init_as_ip=True. Special handling is needed for opset version 8 or lower, because irrespective of user input for keep_initializers_as_inputs, the graph must follow ONNX IR v3 semantics, i.e. all initializers must be listed as ONNX graph input. FaSetting 'keep_initializers_as_inputs=False' for opset version8 or lower would lead to an invalid ONNX graph. Therefore, 'keep_initializers_as_inputs=False' is ignored during export.Exported model will have initializers as graph inputs (compliant to ONNX IR v3).TN)r9r:r3OperatorExportTypesr)rWrSrTval_keep_init_as_ipr-r-r._decide_keep_init_as_inputs rcCs td||S)Nadd_node_names)r)rrSr-r-r._decide_add_node_names(s rcCs2td||}|r.|dk r.|tjjk r.td|S)NrUaIt is recommended that constant folding be turned off ('do_constant_folding=False') when exporting the model in training-amenable mode, i.e. with 'training=TrainingMode.TRAIN' or 'training=TrainingMode.PRESERVE' (when model is in training mode). Otherwise, some learnable model parameters may not translate correctly in the exported ONNX model because constant folding mutates model parameters. Please consider turning off constant folding or setting the training=TrainingMode.EVAL.)rr3r4r<r9r:)rUrSr0r-r-r._decide_constant_folding/s rzinspect.SignaturecCs*t|d|}t|rt|StddS)Nforwardz/model has no forward method and is not callable)getattrcallableinspect signature ValueError)r=Zshould_be_callabler-r-r. _signatureBs  rc Csz t|}Wn:tk rF}zt|d|WYSd}~XYnXzt|j}|ddkrp|dd}i}t|tr|}nt|trt|}n|g}t|dt r|d}|dd}t |}||dD]>}||kr| ||q|j|} | j | j kr| | j qt|tr$|nt|}WnXtk rNtdYn:tk r}ztd|jdW5d}~XYnX|S)Nz, skipping _decide_input_formatrselfrz,No input args, skipping _decide_input_formatzSkipping _decide_input_format )rrr9r:r parametersrr2tuplerlenrfdefaultempty IndexError ExceptionrN) r=rNsigeZordered_list_keysZ args_dictZ args_listZ n_nonkeywordZ optional_argparamr-r-r._decide_input_formatJs>         (rcCsTt|tjr|f}tjj||dddd\}}}t|t||id}|rP||fS|S)NFTstrictZ_force_outplaceZ_return_inputs_states)r)r2r?Tensorr@_get_trace_graphr$r)funcrNrSZ return_outs trace_graph torch_out inputs_statesr-r-r._tracers  rcCsttj|}t}tdtjj||dddd\}}}t|t||tj|krltd||fS)NFTrzXstate_dict changed after running the tracer; something weird is happening in your model!) r?r@_unique_state_dictrZis_autocast_cache_enabledZset_autocast_cache_enabledrr$r~)r=rNZorig_state_dict_keysZprev_autocast_cache_enabledrrrr-r-r._trace_and_get_graph_from_models"   rcCs^g}t||D]F\}}dt|krJtj|\}}|t|q||dk q|S)NZ PackedParams) rrkrr6r?r@_flattenrfr) method_graph args_paramsparam_count_listZinput_Z arg_params_in_vars_r-r-r._get_param_count_listsrcsRtjfddt|}t|t|}|dks:t|rNtd|ddS)zAtorch.jit._flatten removes None. Check if it did so in this case.c3sZt|ttfr(|D]}|EdHqn.t|trP|D]}|EdHq:n|VdSrK)r2rrrvalues)xinnerflattenr-r.rs  z._check_flatten_did_not_remove..flattenrzargs contained z None's after flattening. When exporting a ScriptModule or ScriptFunction, no args may be None because that breaks type propagation.N)rbeartyperrAssertionErrorr)originalZ jit_flattenedZflattened_with_noneZnum_noner-rr._check_flatten_did_not_removes   rz Sequence[Any]zJTuple[_C.Graph, List[_C.IValue], Optional[Any], Optional[_C.ScriptModule]])r=rNr+c Cst|tjjtjjfrLttjt|d}t||d}t|tjjrz |jj }Wn,t k r}zt d|W5d}~XYnXt |t jtt j|jdd}t |\}}|dj } t|t|} t| | } tj| \} } t | t| | dd}||||fSg}|j }t |t||} t ||| dd}|||dfSt||\}}t |tj|}t|}t|}t|t|}t|}t|D]&\}}||kr||||qt ||||dfS)Nrz('forward' method must be a script methodT)ZpreserveParametersrF)r2r?r@rA ScriptModulerrrrroAttributeErrorr~rZ$_jit_pass_onnx_function_substitutionZ_freeze_modulerZ_cZ_jit_onnx_list_model_parametersZ _get_methodrZ"_propagate_and_assign_input_shapesrrtrrrrkrr enumerate setDebugName)r=rNZflattened_argsrrorZfreezed_modulerFparamsrrrrrZ state_dictZ graph_inputsZuser_input_num param_namesiZinpr-r-r._create_jit_graphsf                 rcCs<dd|D}|t|t|d}tt||}|S)NcSsg|] }|qSr- debugName).0rlr-r-r. sz)_get_named_param_dict..)rkrrr)rorZinput_and_param_namesr _params_dictr-r-r._get_named_param_dictsrcCsdt|}i}|r4t|dtr4|d}|dd}|||}t|trP|g}nt|ts`|f}|S)Nr)copydeepcopyr2rrr)r=rNZ input_argsZ input_kwargsexample_outputsr-r-r._get_example_outputs s     rcCst|tjr|jtkr|}|r6tj|tjdntj|tj d}|rbtj| tj dntj| t|jd}|||}|j t|jd}|||fS|fSdS)N)dtype) r2r?rr_qtype_vtype_mapZ dequantizeZtensorq_scaledoubleZfloat32 q_zero_pointZint64to)valueZcast_onnx_acceptedZq_value_dequantizerrZq_valuer-r-r.r%"s  cCs@tddt|dtDs.tdd|Dr>sz)_pre_trace_quant_model..rBcss|]}t|ddVqdSZ is_quantizedFNr)rargr-r-r.r@s)anyrrr?r@trace)r=rNr-r-r._pre_trace_quant_model7s rzTuple[_C.Graph, Dict[str, torch.Tensor], Optional[Union[torch.Tensor, Tuple[torch.Tensor, ...], List[torch.Tensor], Dict[str, torch.Tensor], Any]]]c  Cs`t|tjtttfr|f}t||}t||\} } } }t| | }zt | ||||| ||d} Wn2t k r}ztj d| W5d}~XYnXt|tj jtj jf}|rt||}d}|D]}|t|7}qtj |\}}t| ||tj|tjn\t| ttfs| g}n| }tj t|\}}tdd|DsTt| ||tj|tjt| ||t| | }|rtjtjkr| dks| tjj krt!| |}t"| |tj}t#| tjrzt$| |tjWn<t%k r}ztj&r|j'ddkrW5d}~XYnXt(| |}tjd kr8t)| t*|}t+| t,| || || fS) aConverts model into an ONNX graph. Returns: graph: A TorchScript IR Graph with ONNX nodes. params_dict: Dict from input param name to param value. torch_out: The output tensors resulting from the trace of ``model``. If ``model`` is a :class:`torch.jit.ScriptModule` or :class:`torch.jit.ScriptFunction`, this will be None, since we are not doing any tracing. )rprqrrVrQrFzTorch IR graph at exception: Nr-css|]}t|ddVqdSrr)routr-r-r.rsz"_model_to_graph..rrrr)-r2r?rrfloatr*rrrrrrLlogr@rArrr%rrZ"_jit_pass_onnx_assign_output_shaperr{r8rrr_set_input_and_output_namesrZONNX_CONSTANT_FOLDING_MIN_OPSETr3r4r<Z_jit_pass_onnx_eval_peepholeZ_jit_pass_onnx_constant_foldr|r}r~rrNZ%_jit_pass_onnx_eliminate_unused_itemsZ,_jit_pass_onnx_cast_all_constant_to_floatingZ%_jit_pass_filter_non_tensor_argumentsZ#_jit_decay_packed_param_input_types_apply_friendly_debug_names)r=rNrJrQrRrSrUrprqr0rVrorrrFrrZ is_scriptrZexample_outputs_finalZexample_outputZout_varsdescZoutput_wrappedZoutput_tensorsZout_descrr-r-r._model_to_graphEs&               rc Cs| dkrtj} | dkri} | t_|t_t|||rt| || }t| |}t|||}t ||}t |||||||||d \}}}| || d|| || |W5QRSQRXdS)a Similar to :func:`export`, but returns a text representation of the ONNX model. Only differences in args listed below. All other args are the same as :func:`export`. Args: add_node_names (bool, default True): Whether or not to set NodeProto.name. This makes no difference unless ``google_printer=True``. google_printer (bool, default False): If False, will return a custom, compact representation of the model. If True will return the protobuf's `Message::DebugString()`, which is more verbose. Returns: A UTF-8 str containing a human-readable representation of the ONNX model. N)r0rVF) rONNX_DEFAULT_OPSETrr8rSr!rrrrrZ_pretty_print_onnx)r=rNrPrJr0rQrRrS export_typeZgoogle_printerrTrWrXrrUrVrval_add_node_namesval_do_constant_foldingrorrr-r-r.r&sV$  zTuple[_C.Graph, List[str]])r0rTr+c Cs|ptj}|t_znt||ddVt||}t||}t||\}}}}t |t ||t |t |W5QRXWn.t k r}ztd|W5d}~XYnXg}|D]6} | } | drqtj| d|s|| q||fS)aReturns an approximated list of all ops that are yet supported by :mod:`torch.onnx`. The list is approximated because some ops may be removed during the conversion process and don't need to be converted. Some other ops may have partial support that will fail conversion with particular inputs. Please open a Github Issue for op support requests. Args: model: Same as the `model` parameter in :func:`torch.onnx.export`. args: Same as the `args` parameter in :func:`torch.onnx.export`. training: Same as the `training` parameter in :func:`torch.onnx.export`. opset_version: Same as the `opset_version` parameter in :func:`torch.onnx.export`. Returns: The JIT graph and a list of unconvertible ops in the format of "domain::op". FrIz_Failed to discover unconvertible ops because of errors during the JIT graph generation process.N)zonnx::zprim::r)rrrr8r!rrrrrsrurzr|rrOnnxExporterErrorrcr\ startswithrregistryis_registered_oprstriprf) r=rNr0rTrorrFrZunsupported_opsr`Z domain_opr-r-r.r'$s8         z.Union[torch.nn.Module, torch.jit.ScriptModule]zSet[str])r=rYr+csfdd}dddddfddD}|tjj_t|trZ|rZd d |D}n0t|tr|rd d fd d |D}nt}|r||S)NcsBdfdd}fdd}D]}||||q$dS)NZ _onnx_attrscst|t|dSrK)setattr_get_module_attributes)rFrm attr_namer-r.)_track_module_attributes_forward_pre_hookqszm_setup_trace_module_map..__register_attribute_hook.._track_module_attributes_forward_pre_hookcsJt}|sdS|}i}t|r:t|}t|t||dSrK)rZ_get_tracing_stateror7rdelattrZ%_jit_pass_onnx_track_scope_attributes)rFrmr]Z tracing_stateroZ onnx_attrsrr-r.%_track_module_attributes_forward_hookts   zi_setup_trace_module_map..__register_attribute_hook.._track_module_attributes_forward_hook)rBZregister_forward_hookZregister_forward_pre_hook)rrrr>rr.__register_attribute_hookns    z:_setup_trace_module_map..__register_attribute_hookr)qualified_namer+cSsF|d}ttt|D]&\}}|sd||dSq|S)a Parse qualified variable name and return the unqualified version. Pure numeric atoms are considered inadequate, so this function will look past them, and start from the first non-numeric atom. Example: >>> _unqualified_variable_name('__main__.Foo.bar') 'bar' >>> _unqualified_variable_name('__main__.Foo.bar.0') 'bar.0' .N)splitreversedrr isnumericjoin)rZ name_atomsrZatomr-r-r._unqualified_variable_names z;_setup_trace_module_map.._unqualified_variable_namec s.i|]&\}}|tjtt||qSr-)r?rZ _jit_onnx_create_full_scope_nametypenamer6)rZ_n_m)rr-r. s  z+_setup_trace_module_map..cSsh|]}tt|qSr-)r?rr6)rrFr-r-r. sz*_setup_trace_module_map..cSs*t|trt|Stdt|jdS)NznOnly type of the `nn.Module` should be passed in the set for argument `export_modules_as_functions`. Got `%s`.)r2r6r?rr~rD)vr-r-r._find_typenames  z/_setup_trace_module_map.._find_typenamecsh|] }|qSr-r-)rr )r r-r.rs)Z named_modulesr?r@r_trace_module_mapr2r*set)r=rYrZtrace_module_mapZmodule_typenamesr-)r rr=r._setup_trace_module_mapis    r cCsdtjj_tdSrK)r?r@rr rZ"_jit_pass_onnx_clear_scope_recordsr-r-r-r._reset_trace_module_maps rcs@ttttjj}fdd|DfddDS)Ncsg|]}|dqSrK)rErkrr-r.rsz*_get_module_attributes..csi|]}|t|qSr-rr)rFr-r.rsz*_get_module_attributes..)typingget_type_hintsr6r?nnModule)rFZbase_m_annotationsr-)rrFr.rsrc$Cstjdkst| dkrtjj} tjr4tj dt ddt |t j jrJtd|t_| dkr^tj} | tjkrtj d| dtjdtjd |r| d krtd |stjrtjj}ntjj}| t_|t_z"d t_tj}|t_t}t |t j jt jj frt!||}t"|||t#| || }t$||}t%| ||}t |t&rF|}nd }t'||}| dkrbi} t(| |||t)|||||||||| d \}}}| tjjk }|dkri}t*+|i}|rt*,||t-|.}| d k rt*/||t0|dd}t*1||r0|2|| | ||| ||||| \}} }!}"n(|2i| | d|| ||||| \}} }!}"t34||}|rxt j56d|t37||| | |tjjkr|!szt*8|Wn0t9k r}#zt:|#|#W5d}#~#XYnXW5QRXW5tjstdt_|t_tX|S)NFzCaffe2 ONNX exporter is deprecated in version 2.0 and will be removed in 2.2. Please use PyTorch 2.1 or older for this capability.)category stacklevelztorch.nn.DataParallel is not supported by ONNX exporter, please use 'attribute' module to unwrap model from torch.nn.DataParallel. Try torch.onnx.export(model.module, ...)z Exporting to ONNX opset version zT is not supported. by 'torch.onnx.export()'. The highest opset version supported is z. To use a newer opset version, consider 'torch.onnx.dynamo_export()'. Note that dynamo_export() is in preview. Please report errors with dynamo_export() as Github issues to https://github.com/pytorch/pytorch/issues.rz`export_modules_as_functions` is not supported for `opset_version` < 15.This is because `opset_version` < 15 implies IR version < 8, which means no local function support. T)rqr0rVr0zExported graph: );rr,rrZ ExportTypesZ PROTOBUF_FILEr3rr9r: FutureWarningr2r?rZ DataParallelrr{rrZ#ONNX_TORCHSCRIPT_EXPORTER_MAX_OPSETrZOnnxExporterWarningrONNX_ATEN_FALLBACKrr8rSrZrr rr@rr r!rrrrr_validate_dynamic_axesrrr|Z"_jit_pass_onnx_function_extractionrrZ'_jit_pass_onnx_deduplicate_initializersrZ5_jit_pass_onnx_assign_scoped_names_for_node_and_valueZ _export_onnxrZ_add_onnxscript_fnrLrZ _export_fileZ_check_onnx_protor~Z CheckerError)$r=rNrOrPrJr0rQrRrSrrTrUrVrWrqrXrr{rYrZZ_autograd_inlining_previousZ'module_typenames_to_export_as_functionsrrrZmodel_file_locationrorrZdefer_weight_exportZnode_attr_to_nameprotoZ export_mapZval_use_external_data_formatZ node_namesrr-r-r.r[s.              , r[cCsp|D]b}|D]T}|}|t|kr2q|d|}||||kr||||<qqdS)Nr)rcrkrruniquer\rrE)rornr Zold_namenew_namer-r-r.rs   rcs>tjfdd}|t|d|t|ddS)Ncs|dkr dSt|t|kr8td|t||t|ft}tt||D]\}\}}|dkr||krd}||||| |  || | }||||krL||qLdS)Nz.set_namesrmr])rrrrkoutputs)rorQrRr&r-r%r.rsrc Cspz,tj||tjdtid}||f|WStk rj}z |jdd|df|_W5d}~XYnXdS)z\ This trampoline function gets invoked for every symbolic method call from C++. NrorjZopset original_noderenvrz (occurred when translating ))r GraphContextrjrr8rr5rN)riop_name symbolic_fnrN graph_contextrr-r-r._run_symbolic_methodsr0z_C.Nodez_C.Block)r`r+cCs|SrK)ZaddBlockr`r-r-r. _add_blocksr2rjcCs|SrK)ZaddInputToBlockr3r-r-r._add_input_to_blocksr4z_C.Value)rjrr+cCs ||SrK)ZregisterOutput)rjrr-r-r._add_output_to_blocksr5r)r$rTrScCsdtj||}|tjjk}|tjjk}tj}|ds:dS|rP|sF|r`|s`dSn|s\|r`|s`dSdS)Naten::FT) rrrr3rZ ONNX_ATENrrr)r$rTrSZis_exportable_aten_opZis_onnx_aten_exportZis_aten_fallback_exportZis_caffe2_buildr-r-r._should_aten_fallbacks,   r7r)r.r+cCsPtt|j}|sdS|dj}t|}||kr.wrapper)r:r9r:r)r.r=r-r<r._symbolic_context_handler's r>)r r+cCs*|}|drtrdSt|jS)Nr6r)schemarrrwrZ parse_schema overload_name)r r?r-r-r._get_aten_op_overload_name@srArzDict[_C.Value, _C.Value]z7Optional[Union[_C.Value, Sequence[Optional[_C.Value]]]])rorjr`rkr*r+c sTtj}}|dr&|dd}n|}t|\} } tj|||t|d} t|||rfdd D} } | | d<| j | f|dt i| Szt r|d krtd || d krt rd }n| }|d | }tj|}|dk r6||}|dk r6fd d D} || f|| WSfdd D} | dkrv| j| f|| d iWSt|||r|ndWntk r|tjjkrYdS|tjjkr t s fdd D} | j | f|dt i| YSYn@tk rN}z |jdd| df|_W5d}~XYnXdS)zRuns a symbolic function. The function is used in C++ to export the node to ONNX. Returns: A single or a tuple of Values. None when the node gets cloned as is into the new graph. rNrr(cs,i|]$}|d|dt|qSrrZkindOfr _node_getrr1r-r.rrsz*_run_symbolic_function..r'r@rZcaffe2Z quantized::csi|]}|t|qSr-)rrDrr1r-r.rscs,i|]$}|d|dt|qSrBrCrr1r-r.rsrLcs,i|]$}|d|dt|qSrBrCrr1r-r.rsrz (Occurred when translating z).) rr8r\endswithrparse_node_kindr,rr7ZattributeNamesZ outputsSizeZaten_oprArrwrZregister_quantized_opsrrZget_function_groupgetoprZUnsupportedOperatorErrorZget_min_supportedr~r3rZONNX_FALLTHROUGHrr5rN)rorjr`rkr*rSrTZ node_kindZ ns_op_name namespacer-r/attrsr'domainZsymbolic_function_nameZsymbolic_function_groupr.rr-r1r._run_symbolic_functionIs               rM) symbolic_namecCsNtd|std|dt|\}}|dkrJtd|d|ddS)Nz+^[a-zA-Z0-9-_]+::[a-zA-Z-_]+[a-zA-Z0-9-_]*$zFailed to register operator z. The symbolic name must match the format domain::name, and should start with a letter and contain only alphanumerical charactersrLz. z domain cannot be modified.)rematchrrrrGr)rNnsrr-r-r._verify_custom_op_names  rRrNr.rTcCs6|drd|}t|tj||tgd|dS)aCRegisters a symbolic function for a custom operator. When the user registers symbolic for custom/contrib ops, it is highly recommended to add shape inference for that operator via setType API, otherwise the exported graph may have incorrect shape inference in some extreme cases. An example of setType is `test_aten_embedding_2` in `test_operators.py`. See "Custom Operators" in the module documentation for an example usage. Args: symbolic_name (str): The name of the custom operator in "::" format. symbolic_fn (Callable): A function that takes in the ONNX graph and the input arguments to the current operator, and returns new operator nodes to add to the graph. opset_version (int): The ONNX opset version in which to register. rEaten)ZdecorateN)rrRrZcustom_onnx_symbolicr>rSr-r-r.r(s  rNrTcCs.|drd|}t|tj||dS)a-Unregisters ``symbolic_name``. See "Custom Operators" in the module documentation for an example usage. Args: symbolic_name (str): The name of the custom operator in "::" format. opset_version (int): The ONNX opset version in which to unregister. rErTN)rrRrr unregisterrUr-r-r.r)s  c Cs8t|dkrdSt|drj|dks.t|dkrBdd|jD}|dksVt|dkrjdd|jD}t|prg|pxg}|D]\}}||krtd|dt |t rtd |i}t |D]Z\}} t | t st d | |kr td | d |d qt|dt|d|| <q|||<qdS)z=Ensures dynamic axes argument is follows the expected format.rNrocSsg|] }|qSr-r)rrr-r-r.rsz*_validate_dynamic_axes..cSsg|] }|qSr-r)ryr-r-r.rsz Provided key z2 for dynamic axes is not a valid input/output namezNo names were found for specified dynamic axes of provided input.Automatically generated names will be applied to each dynamic axes of input z3The type of axis index is expected to be an integerzDuplicate dynamic axis index z was provided for input rZ_dynamic_axes_r)rr7rorkr'r rCr9r:r2rrrrr) rVr=rQrRZ valid_namesrGrZ value_dictrrr-r-r.r s:      rz Union[torch.nn.Module, Callable])r=r+cCstt|tjjr|jn|SrK)rrr2r?rrrr>r-r-r.r#6s)FFNNNN)F)T)n__doc__ __future__r contextlibrriorOrxrr9rrrrrr r r r r rrrr?Ztorch._C._onnxrZ_onnxr3Ztorch.jit._traceZtorch.serializationZ torch.onnxrrrrrZtorch.onnx._globalsrZtorch.onnx._internalrrrrr__all__rrcontextmanagerrrrr r!r4r<rrr"rbrerr$rrrrrrrrrrrrrZquint8Zuint8Zqint8Zint8Zqint32Zint32Zquint4x2rr%rrZ_disable_dynamor&r'r rrr[rrr0r2r4r5r7r:r>rArMrRr(r)rr#r-r-r-r.s <   /   2\    +   '    3    LDM   ]   v$ ,