U ,-eQ @sddlZddlmZddlmZmZddlmZddlm Z m Z m Z m Z ddl mZmZddlmZmZddlmZdd lmZmZmZed Zd d d ddddddg ZGdddeZeeedddZedddZddZeee e ee ee efddd Z d@eeee eed!d"d#Z!ee"ee#d$d%d&Z$ee"ed'd(d)Z%dAeeee"e ee#ed+d,d-Z&eed.d/d0Z'eed.d1d2Z(ed3d4d5Z)e*d6kreZ+e+,Z-ee-j./e-_.ze0d7e&e-j1e-j2e-j.e-j3e-j4e-j5e-je-j(r,eee-j1d8kr e0d9e0d:e'e-j.e-_6e(e-j6e-_7e-j8rre0d;e)e-j.e9e-d<r\e)e-j6e9e-d=rre)e-j7Wn:e:k rZ;ze0d>e;e= than 2Gbz --quantizez/Quantize the neural network to be run with int8output)super__init__ add_argumentstrSUPPORTED_PIPELINESint)self __class__c/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/transformers/convert_graph_to_onnx.pyr 3sJ z$OnnxConverterArgumentParser.__init__)__name__ __module__ __qualname____doc__r __classcell__r(r(r&r)r.sr)filename identifierreturncCs|j|j||jS)aE Append a string-identifier at the end (before the extension, if any) to the provided filepath Args: filename: pathlib.Path The actual path object we would like to add an identifier suffix identifier: The suffix to add Returns: String with concatenated identifier at the end of the filename )parentjoinpathstem with_suffixsuffix)r/r0r(r(r)generate_identified_filename\s r7)minimum_versioncCsXz6ddl}t|j}|tkr4td|jd|dWntk rRtdYnXdS)z Check onnxruntime is installed and if the installed version match is recent enough Raises: ImportError: If onnxruntime is not installed or too old version is found rNz*We found an older version of onnxruntime (z&) but we require onnxruntime to be >= zp to enable all the conversions options. Please update onnxruntime by running `pip install --upgrade onnxruntime`zonnxruntime doesn't seem to be currently installed. Please install the onnxruntime by running `pip install onnxruntime` and relaunch the conversion.) onnxruntimer __version__ORT_QUANTIZE_MINIMUM_VERSION ImportError)r8r9Z ort_versionr(r(r)check_onnxruntime_requirementsis r=cCs|td|jjj}gg}}|ddD]8}||krN|||||q(t|dqbq(td||t|fS)a Ensure inputs are presented in the correct order, without any Non Args: model: The model used to forward the input data tokens: BatchEncoding holding the input data input_names: The name of the inputs Returns: Tuple z$Ensuring inputs are in correct orderNz, is not present in the generated input list.zGenerated inputs order: )printforward__code__ co_varnamesappendtuple)modeltokens input_namesZmodel_args_name model_argsordered_input_namesarg_namer(r(r)ensure_valid_inputs    rK)nlp frameworkr1c stttdfdd |jd|d}|jjd|dkrD|jf|n||}t|tr`| }t|t t fst|f}t | }fdd | D}g}|D](}t|t t fr||q||qd d tt|D}fd d t||D} t|f| } ||| |fS) a? Attempt to infer the static vs dynamic axes for each input and output tensors for a specific model Args: nlp: The pipeline object holding the model to be exported framework: The framework identifier to dispatch to the correct inference scheme (pt/tf) Returns: - List of the inferred input variable names - List of the inferred output variable names - Dictionary with input/output variables names as key and shape tensor as value - a BatchEncoding reference which was used to infer all the above information )nameis_inputseq_lencst|ttfr&fdd|DSddt|jDddi}rvt|jdkr^d|d<qtd t|jd n,fd dt|jD}|d d |Dtdrdnddd||S)Ncsg|]}|qSr(r().0t)build_shape_dictrOrNrPr(r) sz:infer_shapes..build_shape_dict..cSsg|]\}}|dkr|qS)r>r()rQZaxisZnumelr(r(r)rTsrbatchsequencer>zUnable to infer tensor axes ()csg|]\}}|kr|qSr(r()rQdimshape)rPr(r)rTscSsi|] }|dqS)rWr()rQrYr(r(r) sz:infer_shapes..build_shape_dict..zFound inputr z with shape: ) isinstancerDlist enumeraterZlen ValueErrorupdater?)rNtensorrOrPZaxesZseq_axes)rS)rOrNrPr)rSs "z&infer_shapes..build_shape_dictzThis is a sample output)Zreturn_tensorsrcs i|]\}}|||dqS)Tr(rQkvrSrPr(r)r[sz infer_shapes..cSsg|]}d|qS)Zoutput_r()rQir(r(r)rTsz infer_shapes..cs i|]\}}|||dqS)Fr(rfrir(r)r[s)r"boolr$ tokenizerZ input_idsrZrEr^rZto_tupler_rDkeysitemsextendrCrangerazipdict) rLrMrFoutputsZ input_varsZinput_dynamic_axesZ outputs_flatr output_namesZoutput_dynamic_axes dynamic_axesr(rir) infer_shapess&      rv) pipeline_namerMrErlr1cKs`|dkr |}|dkr"ts"td|dkr8ts8tdtd|d|dt|||||d S) a Convert the set of arguments provided through the CLI to an actual pipeline reference (tokenizer + model Args: pipeline_name: The kind of pipeline to use (ner, question-answering, etc.) framework: The actual model to convert the pipeline from ("pt" or "tf") model: The model name which will be loaded by the pipeline tokenizer: The tokenizer name which will be loaded by the pipeline, default to the model's value Returns: Pipeline object NrLCannot convert because PyTorch is not installed. Please install torch first.rLCannot convert because TF is not installed. Please install tensorflow first.zLoading pipeline (model: z , tokenizer: rX)rErlrM model_kwargs)r Exceptionrr?r )rwrMrErlZ models_kwargsr(r(r)load_graph_from_argssr|)rLopsetruse_external_formatc Cststdddl}ddlm}ddlm}td|j| pt |d\}}} } t |j | |\} } |r||j | | | || d|d|d n||j | | | || d|d W5QRXdS) a Export a PyTorch backed pipeline to ONNX Intermediate Representation (IR Args: nlp: The pipeline to be exported opset: The actual version of the ONNX operator set to use output: Path where will be stored the generated ONNX model use_external_format: Split the model definition from its parameters to allow model bigger than 2GB Returns: rxrN)export)is_torch_less_than_1_11zUsing framework PyTorch: rT)frGrtrudo_constant_foldingZuse_external_data_formatZenable_onnx_checker opset_version)rrGrtrurr)rr{torchZ torch.onnxrZtransformers.pytorch_utilsrr?r:Zno_gradrvrKrEas_posix) rLr}rr~rrrrGrtrurFrIrHr(r(r)convert_pytorchs@     r)rLr}rc ststdtdzddlddl}ddlm}tdjjd|t|d\}}}}|j |j fd d | D} |j j|j | ||d \} } Wn@tk r} z"td | jd | jd| W5d} ~ XYnXdS)av Export a TensorFlow backed pipeline to ONNX Intermediate Representation (IR) Args: nlp: The pipeline to be exported opset: The actual version of the ONNX operator set to use output: Path where will be stored the generated ONNX model Notes: TensorFlow cannot export model bigger than 2GB due to internal constraint from TensorFlow ryzD/!\ Please note TensorFlow doesn't support exporting model > 2Gb /!\rN)r:zUsing framework TensorFlow: z , tf2onnx: rcs g|]\}}jj||dqS))rN)Z TensorSpecZ from_tensor)rQkeyrdrr(r)rTUsz&convert_tensorflow..)r}Z output_pathzCannot import z6 required to convert TF model to ONNX. Please install z first. )rr{r?Z tensorflowtf2onnxr:versionVERSIONrvrEZpredictdatarnconvertZ from_kerasrr<rN) rLr}rrZt2ovrGrtrurFZinput_signatureZ model_proto_er(rr)convert_tensorflow8s*  rF)rMrErr}rlr~rwc Kstdttd|t||||f|}|jsVtd|jt|jn,t t |jdkrt d|jd|dkrt ||||n t |||dS) a Convert the pipeline object to the ONNX Intermediate Representation (IR) format Args: framework: The framework the pipeline is backed by ("pt" or "tf") model: The name of the model to load for the pipeline output: The path where the ONNX graph will be stored opset: The actual version of the ONNX operator set to use tokenizer: The name of the model to load for the pipeline, default to the model's name if not provided use_external_format: Split the model definition from its parameters to allow model bigger than 2GB (PyTorch only) pipeline_name: The kind of pipeline to instantiate (ner, question-answering, etc.) model_kwargs: Keyword arguments to be forwarded to the model constructor Returns: zoThe `transformers.convert_graph_to_onnx` package is deprecated and will be removed in version 5 of TransformerszONNX opset version set to: zCreating folder rzFolder z" is not empty, aborting conversionrN)warningswarn FutureWarningr?r|r2existsrrrarr{rr) rMrErr}rlr~rwrzrLr(r(r)r`s r)onnx_model_pathr1cCsTddlm}m}t|d}|}||_|||}td|dtd|S)a> Load the model at the specified path and let onnxruntime look at transformations on the graph to enable all the optimizations possible Args: onnx_model_path: filepath where the model binary description is stored Returns: Path where the optimized model binary description has been saved rInferenceSessionSessionOptionsz -optimizedz$Optimized model has been written at : ✔zY/!\ Optimized model contains hardware specific operators which might not be portable. /!\)r9rrr7rZoptimized_model_filepathr?)rrrZopt_model_pathZ sess_optionrr(r(r)optimizes   rc Csddl}ddl}ddlm}ddlm}ddlm}ddlm }| | }t |j t dkrhtd|}||t |j t d kr||d d |jd d d dddt|d } n$||d d |jd d d dddt|d } | t|d} td| d|| jj| | S)z Quantize the weights of the model from float32 to in8 to allow very efficient inference on modern CPU Args: onnx_model_path: Path to location the exported ONNX model is stored Returns: The Path generated for the quantized rN) ModelProto)QuantizationMode) ONNXQuantizer)IntegerOpsRegistryz1.5.0zpModels larger than 2GB will fail to quantize due to protobuf constraint. Please upgrade to onnxruntime >= 1.5.0.z1.13.1FT) rE per_channel reduce_rangemodestatic weight_qTypeZ input_qType tensors_rangenodes_to_quantizenodes_to_excludeop_types_to_quantize) rErrrrrZactivation_qTyperrrrz -quantizedz$Quantized model has been written at r)onnxr9Z onnx.onnx_pbrZonnxruntime.quantizationrZ'onnxruntime.quantization.onnx_quantizerrZ!onnxruntime.quantization.registryrloadrr r:r?ZCopyFromZ IntegerOpsr_Zquantize_modelr7Z save_modelrE) rrr9rrrrZ onnx_modelZ copy_modelZ quantizerZquantized_model_pathr(r(r)quantizesZ       r)pathc Csddlm}m}ddlm}td|dz.|}|||dgd}td|d Wn2|k r}ztd |d W5d}~XYnXdS) Nrr)RuntimeExceptionz"Checking ONNX model loading from: z ...ZCPUExecutionProvider) providerszModel u correctly loaded: ✔zError while loading the model u: ✘)r9rrZ+onnxruntime.capi.onnxruntime_pybind11_staterr?r)rrrrZ onnx_optionsrrer(r(r)verifys r__main__z' ====== Converting model to ONNX ======raV Using TensorFlow might not provide the same optimization level compared to PyTorch. For TensorFlow users you can try optimizing the model directly through onnxruntime_tools. For more information, please refer to the onnxruntime documentation: https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/python/tools/transformers z$ ====== Optimizing ONNX model ======z+ ====== Check exported ONNX model(s) ======optimized_outputquantized_outputz"Error while converting the model: r>)N)NFr)=rargparserosrrpathlibrtypingrrrr Zpackaging.versionr r Ztransformers.pipelinesr r Ztransformers.tokenization_utilsrZtransformers.utilsrrrr;r#rr"r7r=rKrvr|r$rkrrrrrrr*parser parse_argsargsrabsoluter?rMrEr}rlr~rrZ check_loadinghasattrr{rexitr(r(r(r)s    . &A 7- 2K