U ,-em@sUddlZddlZddlmZddlmZmZddlmZddl m Z m Z ddl m Z mZddlmZmZmZddlmZddlmZdd l mZdd lmZdd lmZmZdd lmZmZd dl m!Z!d dl"m#Z#ddl$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-ddl.m/Z/ddl0m1Z1ddl2Z2ddl3Z3dddddddddddddd d!d"d#d$d%d&d'd(gZ4ejj5j6ejj5j7ejj5j8hZ9e/Gd)d(d(Z:ee)e;d*d+d$Zd/dZ?e%e%d0d1dZ@eAe%d2d3dZBee&e'ed4d5dZCee'eed6d7d ZDejjEe)d8d9dZFejjEeeAe)ed:d;dZGee(eAejjEfe(ee;fe;d<d=dZHee'eId4d>dZJe'eIe%ege'eIfd?d@d'ZKe1d"dAZLeLdBdCej;eKd geMeKdDgieLdBdEeIeJieLdBdFeIeJieLdBdGeIeJieLdBdHeIeKd gieLdBdIeIeJieLdBejNeIeJieLdBdJeIeKd gieLdBdKeIeKd gieLdBejOeIeKd gieLdBdLeIeJii ZPe(eLe(e,eQejRfe%ege'eIfffeSd%<iZTe(e,eQejRfe%ege'eIffeSd<ee(e,eQejRfe%ege'eIffd4dMdZUdoee(eAejEfe&e-ejEe)e&edNdOd!ZVee+e)dPfeedQdRdZWe!e;e+e'eAe'e-e)fdSdTdZXdpee(eAejjEfee&e)e;dUdVdWZYdqee(eAejjEfee&e)e;dUdXdYZZee(eAejjEfe&ejjEdZd[d\Z[eejjEe(eAejjEfeed]d^d_Z\eejjEe(eAejjEfeed]d`daZ]ee(eAejjEfe&edbdcddZ^ededfdgZ_e,ee fedhdidjZ`dreee;e;dldmdnZadS)sN) QConfigAny QuantType)DTypeWithConstraints)FakeQuantizeBaseFixedQParamsFakeQuantize)FixedQParamsObserver ObserverBase)float16_static_qconfigfloat16_dynamic_qconfigqconfig_equals) DeQuantStub)"activation_is_statically_quantized)_is_activation_post_process)QConfigMapping) GraphModulemap_arg)GraphNode)PrepareCustomConfig)quantized_decomposed_lib) CallableOptionalListDictAnySetTupleUnionType) dataclass) namedtupleall_node_args_except_firstall_node_args_have_no_tensorsassert_and_get_unique_devicecollect_producer_nodescreate_getattr_from_value'create_node_from_old_node_preserve_metaEMPTY_ARG_DICTget_custom_module_class_keysget_linear_prepack_op_for_dtypeget_new_attr_name_with_prefix(get_non_observable_arg_indexes_and_typesget_qconv_prepack_op#get_skipped_module_name_and_classes graph_module_from_producer_nodesmaybe_get_next_moduleNodeInfonode_arg_is_biasnode_arg_is_weightNON_OBSERVABLE_ARG_DICTNON_QUANTIZABLE_WEIGHT_OPSreturn_arg_listObservedGraphModuleAttrsc@seZdZUeeefed<eeeeeffed<e ed<eee fed<e ed<e ed<e eed<dZe ed <d Zeeeed <d Zeeeed <d S) r7Znode_name_to_qconfigZnode_name_to_scopeprepare_custom_configZ!equalization_node_name_to_qconfigZqconfig_mappingZis_qatZobserved_node_namesFis_observed_standalone_moduleN&standalone_module_input_quantized_idxs'standalone_module_output_quantized_idxs)__name__ __module__ __qualname__rstrr__annotations__rtyperrrboolrr9r:rrintr;rDrD_/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/torch/ao/quantization/fx/utils.pyr7Ks   )nodeargreturncCsXd}d|jkr |jddd}|dk rH|t|jkrH|j||krHdS|jd|kS)zReturns if node arg is weightNtarget_dtype_info weight_indexTweightmetagetlenargskwargs)rFrGrJrDrDrEr3Xs  $cCsXd}d|jkr |jddd}|dk rH|t|jkrH|j||krHdS|jd|kS)zReturns if node arg is biasNrI bias_indexTZbiasrL)rFrGrRrDrDrEr2as  $)custom_module_mappingrHcCsDt}tjtjtjfD]$}||i}t|}||O}qt|S)a Get all the unique custom module keys in the custom config dict e.g. Input: { QuantType.STATIC: { CustomModule1: ObservedCustomModule }, QuantType.DYNAMIC: { CustomModule2: DynamicObservedCustomModule }, QuantType.WEIGHT_ONLY: { CustomModule3: WeightOnlyObservedCustomModule }, } Output: # extract the keys across all inner STATIC, DYNAMIC, and WEIGHT_ONLY dicts [CustomModule1, CustomModule2, CustomModule3] )setrZSTATICZDYNAMICZ WEIGHT_ONLYrNkeyslist)rSZfloat_custom_module_classesZ quant_modeZquant_mode_custom_module_configZ quant_mode_custom_module_classesrDrDrEr)js    cCs6|tjkrtjjjS|tjkr(tjjjStd|dS)Nz&can't get linear prepack op for dtype:)torchZfloat16ops quantizedZlinear_prepack_fp16Zqint8Zlinear_prepack Exception)dtyperDrDrEr*s     )conv_oprHc Cstjjjtjjjtjjjtjjjtjjj tjjj tjjj tjjj tjjj tjjjtjjjtjjji}||d}|std||S)NzDidn't find prepack op for )rWnn functionalZconv1drXrYZconv1d_prepackZconv2dZconv2d_prepackZconv3dZconv3d_prepackZconv_transpose1dZconv_transpose1d_prepackZconv_transpose2dZconv_transpose2d_prepackZconv_transpose3dZconv_transpose3d_prepackrNAssertionError)r\Z prepack_opsZ prepack_oprDrDrEr-s  )prefixrHcs&ddtjjdfdd }|S)N._)modulecs>tdfdd }d}||}t||r:|d7}||}q|S)Nics t|SN)r?rdr`rDrE get_attr_nameszOget_new_attr_name_with_prefix..get_new_attr_name..get_attr_namerr)rChasattr)rcrhre attr_namergrDrEget_new_attr_names  z8get_new_attr_name_with_prefix..get_new_attr_name)replacerWr]Module)r`rkrDrgrEr+s  rFrHcCs|g}|g}|r|}t|jt|j}|D]H}t|tsDq4|jdkrTdS|||jdkrr|j t ks4||q4q |S)a Starting from a target node, trace back until we hit inpu or getattr node. This is used to extract the chain of operators starting from getattr to the target node, for example def forward(self, x): observed = self.observer(self.weight) return F.linear(x, observed) collect_producer_nodes(observed) will either return a list of nodes that produces the observed node or None if we can't extract a self contained graph without free variables(inputs of the forward function). placeholderN call_function) poprVrPrQvalues isinstanceropappendtargetgetattr)rFnodesZfrontierZall_argsrGrDrDrEr%s    )rootproducer_nodesrHcslt|dkstd|t}ifdd}|D]}||||<q6|||dt||}|S)a- Construct a graph module from extracted producer nodes from `collect_producer_nodes` function Args: root: the root module for the original graph producer_nodes: a list of nodes we use to construct the graph Return: A graph module constructed from the producer nodes rz'list of producer nodes can not be emptycst|fddS)Ncs|SrfrDrFenvrDrEzDgraph_module_from_producer_nodes..load_arg..)rar|rDrEload_argsz2graph_module_from_producer_nodes..load_arg)rOr_reverserZ node_copyoutputr)ryrzgraphrZ producer_nodeZ graph_modulerDr|rEr/s   )rcrHcCs^dd|Ddd|DB}t|dks>td|t|dkrVtt|nd}|S)z Returns the unique device for a module, or None if no device is found. Throws an error if multiple devices are detected. cSsh|] }|jqSrDdevice.0prDrDrE sz/assert_and_get_unique_device..cSsh|] }|jqSrDrrrDrDrErsrzKprepare only works with cpu or single-device CUDA modules, but got devices rN) parametersbuffersrOr_nextiter)rcZdevicesrrDrDrEr$s)rcrr`valuerHc CsZt|}||}t|}t|tjr0|n tj||d}|||| d|}|S)z Given a value of any type, creates a getattr node corresponding to the value and registers the value as a buffer to the module. rget_attr) r+r$rsrWZTensorclonedetachZtensorZregister_buffer create_node) rcrr`rrkrjr new_valueZ attr_noderDrDrEr&s   )rFmodulescacherHc Cs|r||kr||Sd}t|ts*d}n|jdkrUsz=return_arg_list..arg_indices_func..rDr{rr{rEarg_indices_funcTsz)return_arg_list..arg_indices_func)rrrC)rrrDrrEr6Osz op targetrZ masked_fillZpermuterepeatZreshaper transpose unsqueezeZ unsqueeze_viewcCst|j|j}t|tS)z Returns a dict with of non float tensor types as keys and values which correspond to a function to retrieve the list (which takes the node as an argument) )r1rtrvr4rNr()rFinforDrDrEr,s)rFrtarget_module_typetarget_functional_typerHcCsf|jD]V}|jdkr<|dk r