U 9%ehn@sjddlZddlmZmZddlmZmZmZmZm Z m Z m Z ddl Z ddl mZmZmZddlmZmZmZddlmZerdd l mZmZeeZerdd lmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+erdd lm,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4es.es.e5d dd e6e6ee6eegeffdddZ7GdddZ8dS)N)partialreduce) TYPE_CHECKINGCallableDictOptionalTupleTypeUnion)PretrainedConfigis_tf_availableis_torch_available)TF2_WEIGHTS_NAME WEIGHTS_NAMElogging) OnnxConfigPreTrainedModelTFPreTrainedModel) AutoModelAutoModelForCausalLMAutoModelForImageClassificationAutoModelForImageSegmentationAutoModelForMaskedImageModelingAutoModelForMaskedLMAutoModelForMultipleChoiceAutoModelForObjectDetectionAutoModelForQuestionAnswering AutoModelForSemanticSegmentationAutoModelForSeq2SeqLM"AutoModelForSequenceClassificationAutoModelForSpeechSeq2SeqAutoModelForTokenClassificationAutoModelForVision2Seq) TFAutoModelTFAutoModelForCausalLMTFAutoModelForMaskedLMTFAutoModelForMultipleChoiceTFAutoModelForQuestionAnswering"TFAutoModelForSemanticSegmentationTFAutoModelForSeq2SeqLM$TFAutoModelForSequenceClassification!TFAutoModelForTokenClassificationzThe ONNX export features are only supported for PyTorch or TensorFlow. You will not be able to export models without one of these libraries installed.onnx_config_cls)supported_featuresr0returncGsz|dkrtdt}|dD]}t||}qi}|D]>}d|krb|dd}t|j|d||<q6t|j|d||<q6|S)ac Generate the mapping between supported the features and their corresponding OnnxConfig for a given model. Args: *supported_features: The names of the supported features. onnx_config_cls: The OnnxConfig full name corresponding to the model. Returns: The dictionary mapping a feature to an OnnxConfig constructor. Nz#A OnnxConfig class must be provided. -with-past)task) ValueError transformerssplitgetattrreplacerZ with_pastZfrom_model_config)r0r1Z config_cls attr_namemappingfeaturer6r?Y/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/transformers/onnx/features.pysupported_features_mapping7s   rAcC@sHeZdZiZiZer:eeee e e e e eeeeeeedZerXeeeeeeeeed Zeddddddd d edd d d dddddd edddd eddd dddddd eddd dddddd edd d d dddddd edd d d dddd edd d d dddd edd d d dddd eddd dddddd eddd edd dd edddddddd edddd edddddddd edddd edddddd d eddddddd!d eddd"d edd#d$d%d eddddddd&d eddd ddddd'd eddd ddddd(d edd d d ddd)d edd d d ddd*d edd d d dd+d edd,d eddddddd-d eddd.d eddddd/d eddddd0d eddd1d edd ddd2d eddddddd3d edd ddd d d4d edd d d ddddd5d eddddddd6d eddd7d eddd8d eddd9d edd ddd:d edd ddd;d eddd eddd ddddd?d eddd@d eddd dddddAd eddd ddddddBd edddCdDd edddddddEd edddFd edd dddGd edHdId edddJd edd dKdLdMd eddd dddddNd eddd dddddOd edd#dPd dQ;Z e!e"dRdSdTdUe #DZ$e%dqe&e'e&e(e&e)e*ge+ffdWdXdYZ,e%e&e&dZd[d\Z-e%e&d]d^d_Z.e%dre&e&e/dadbdcZ0e%dse&e&e&dddedfZ1e%dte&e&e&e&e2dgdhdidjZ3e%due2dge&e4e&e)fdkdldmZ5e&e&e+dndodpZ6dVS)vFeaturesManager)default masked-lm causal-lm seq2seq-lmsequence-classificationtoken-classificationmultiple-choiceobject-detectionquestion-answeringimage-classificationimage-segmentationz masked-imsemantic-segmentation vision2seq-lm speech2seq-lm) rCrDrErFrGrHrIrKrNrCrDrGrIrHrKzmodels.albert.AlbertOnnxConfigr/zdefault-with-pastrEzcausal-lm-with-pastrFzseq2seq-lm-with-pastzmodels.bart.BartOnnxConfigrLzmodels.beit.BeitOnnxConfigzmodels.bert.BertOnnxConfigz!models.big_bird.BigBirdOnnxConfigz/models.bigbird_pegasus.BigBirdPegasusOnnxConfigz&models.blenderbot.BlenderbotOnnxConfigz1models.blenderbot_small.BlenderbotSmallOnnxConfigzmodels.bloom.BloomOnnxConfigz$models.camembert.CamembertOnnxConfigzmodels.clip.CLIPOnnxConfigz models.codegen.CodeGenOnnxConfigz"models.convbert.ConvBertOnnxConfigz"models.convnext.ConvNextOnnxConfigz&models.data2vec.Data2VecTextOnnxConfigz(models.data2vec.Data2VecVisionOnnxConfigz models.deberta.DebertaOnnxConfigz%models.deberta_v2.DebertaV2OnnxConfigzmodels.deit.DeiTOnnxConfigrJrMzmodels.detr.DetrOnnxConfigz&models.distilbert.DistilBertOnnxConfigz models.electra.ElectraOnnxConfigz"models.flaubert.FlaubertOnnxConfigzmodels.gpt2.GPT2OnnxConfigzmodels.gptj.GPTJOnnxConfigzmodels.gpt_neo.GPTNeoOnnxConfigz"models.groupvit.GroupViTOnnxConfigzmodels.ibert.IBertOnnxConfigz"models.imagegpt.ImageGPTOnnxConfigz"models.layoutlm.LayoutLMOnnxConfigz&models.layoutlmv3.LayoutLMv3OnnxConfigzmodels.levit.LevitOnnxConfigzmodels.longt5.LongT5OnnxConfigz&models.longformer.LongformerOnnxConfigzmodels.marian.MarianOnnxConfigzmodels.mbart.MBartOnnxConfigz&models.mobilebert.MobileBertOnnxConfigz)models.mobilenet_v1.MobileNetV1OnnxConfigz)models.mobilenet_v2.MobileNetV2OnnxConfigz$models.mobilevit.MobileViTOnnxConfigzmodels.mt5.MT5OnnxConfigzmodels.m2m_100.M2M100OnnxConfigzmodels.owlvit.OwlViTOnnxConfigz$models.perceiver.PerceiverOnnxConfigz&models.poolformer.PoolFormerOnnxConfigz models.rembert.RemBertOnnxConfigzmodels.resnet.ResNetOnnxConfigz models.roberta.RobertaOnnxConfigz"models.roformer.RoFormerOnnxConfigrNz$models.segformer.SegformerOnnxConfigz(models.squeezebert.SqueezeBertOnnxConfigzmodels.swin.SwinOnnxConfigzmodels.t5.T5OnnxConfigrOz*zFeaturesManager.ccs|]}|VqdSrQ)keys).0vr?r?r@ *szFeaturesManager.N) model_type model_namer2cCsX|}|tjkrN|r&|d|dn|}t|dttjd|dtj|S)a Tries to retrieve the feature -> OnnxConfig constructor map from the model type. Args: model_type (`str`): The model type to retrieve the supported features for. model_name (`str`, *optional*): The name attribute of the model object, only used for the exception message. Returns: The dictionary mapping each feature to a corresponding OnnxConfig constructor. z ()z is not supported yet. Only z' are supported. If you want to support z) please propose a PR or open up an issue.)lowerrB_SUPPORTED_MODEL_TYPEKeyErrorlistrV)rZr[Zmodel_type_and_model_namer?r?r@%get_supported_features_for_model_type,s z5FeaturesManager.get_supported_features_for_model_type)r>r2cCs |ddS)Nr4r5)r;)r>r?r?r@feature_to_taskFszFeaturesManager.feature_to_task frameworkcCsL|dkrtd|dn.|dkr2ts2tdn|dkrHtsHtddS) z Validates if the framework requested for the export is both correct and available, otherwise throws an exception. pttfzAOnly two frameworks are supported for ONNX export: pt or tf, but z was provided.rfzOCannot export model to ONNX using PyTorch because no PyTorch package was found.rgzUCannot export model to ONNX using TensorFlow because no TensorFlow package was found.N)r7r RuntimeErrorr rcr?r?r@_validate_framework_choiceJs  z*FeaturesManager._validate_framework_choicerf)r>rdr2cCsXt|}t||dkr$tj}ntj}||krPtd|dttj||S)ad Attempts to retrieve an AutoModel class from a feature name. Args: feature (`str`): The feature required. framework (`str`, *optional*, defaults to `"pt"`): The framework to use for the export. Returns: The AutoModel class corresponding to the feature. rfzUnknown task: z. Possible values are )rBrbri_TASKS_TO_AUTOMODELS_TASKS_TO_TF_AUTOMODELSr_r`values)r>rdr6Ztask_to_automodelr?r?r@get_model_class_for_featureYs  z+FeaturesManager.get_model_class_for_feature)modelrdr2cCs|dk r |Sddd}ddd}tj|rtjtj|trHd}n2tjtj|trdd}ntd td td t d ||d n t rd}nt rd}nt dt d||d|S)a Determines the framework to use for the export. The priority is in the following order: 1. User input via `framework`. 2. If local checkpoint is provided, use the same framework as the checkpoint. 3. Available framework in environment, with priority given to PyTorch Args: model (`str`): The name of the model to export. framework (`str`, *optional*, defaults to `None`): The framework to use for the export. See above for priority if none provided. Returns: The framework to use for the export. NZPyTorchZ TensorFlowreZtorchZtf2onnxrfrgzMCannot determine framework from given checkpoint location. There should be a z for PyTorch or z for TensorFlow.zLocal z model found.zKNeither PyTorch nor TensorFlow found in environment. Cannot export to ONNX.zFramework not requested. Using z to export to ONNX.) ospathisdirisfilejoinrrFileNotFoundErrorloggerinforr EnvironmentError)rnrdZ framework_mapZ exporter_mapr?r?r@determine_frameworkts(   z#FeaturesManager.determine_frameworkr)r>rnrd cache_dirr2cCst||}t||}z|j||d}WnRtk r||dkr^td|j|d|d}ntd|j|d|d}YnX|S)a Attempts to retrieve a model from a model's name and the feature to be enabled. Args: feature (`str`): The feature required. model (`str`): The name of the model to export. framework (`str`, *optional*, defaults to `None`): The framework to use for the export. See `FeaturesManager.determine_framework` for the priority should none be provided. Returns: The instance of the model. )ryrfz=Loading TensorFlow model in PyTorch before exporting to ONNX.T)Zfrom_tfryz=Loading PyTorch model in TensorFlow before exporting to ONNX.)Zfrom_ptry)rBrxrmZfrom_pretrainedOSErrorrurv)r>rnrdryZ model_classr?r?r@get_model_from_features    z&FeaturesManager.get_model_from_feature)rnr>r2cCsd|jjdd}t|dd}tj||d}||krNt|jjd|d||jjtj||fS)aR Check whether or not the model has the requested features. Args: model: The model to export. feature: The name of the feature to check if it is available. Returns: (str) The type of the model (OnnxConfig) The OnnxConfig instance holding the model export properties. _-namer5)r[z doesn't support feature z. Supported values are: )configrZr;r:rBrar7r^)rnr>rZr[Zmodel_featuresr?r?r@check_supported_model_or_raises z.FeaturesManager.check_supported_model_or_raise)rZr>r2cCstj||S)aU Gets the OnnxConfig for a model_type and feature combination. Args: model_type (`str`): The model type to retrieve the config for. feature (`str`): The feature to retrieve the config for. Returns: `OnnxConfig`: config for the combination )rBr^)rZr>r?r?r@ get_configs zFeaturesManager.get_config)N)rf)N)NN)rC)7__name__ __module__ __qualname__rjrkrrrrr!r"r$rrrrrrr r%r#r r&r(r'r,r-r.r)r*r+rAr^sortedrrlZAVAILABLE_FEATURES staticmethodstrrrrr rrarbrir rmrxr r{rrrr?r?r?r@rBUs                            5 1  rB)9ro functoolsrrtypingrrrrrr r r8r5r r rutilsrrrrrrrZ get_loggerrruZtransformers.models.autorrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.warningrrArBr?r?r?r@s0$  D,