U 9%etb@sbdZddlZddlZddlmZddlmZmZddlm Z m Z m Z m Z m Z mZddlZddlmZddlmZddlmZee eefe ee eeffe fZd e_e Zee e e e fe e e e e ffe fZd e_Gd d d Zejjejjejjejjejj ejj!ejj"ejj#ejj$ejj%ejj&ejj'ejj(ejj)ejj*hZ+ejj,j-ejj,j.ejj,j/ejj,j0ejj,j1ejj,j2ejj,j3ejj,j4ejj,j5ejj,j6ejj,j7ejj,j8ejj,j9ejj,j:ejj,j;ejj,jejj,j?ej@ejAej?ejBejCejDejEejFejGhZHejId d ddddddddddddddddddd d!d"d#hZJd$d%ZKd&d'ZLd(d)ZMd*d+ZNe eOe d,d-d.ZPd/d0ZQd1d2ZRd3d4ZSd5d6ZTd7d8ZUd9d:ZVd;d<ZWd=d>ZXd?d@ZYdAdBZZdCdDZ[e\dEdFdGZ]dHdIZ^dJdKZ_ej`ej`e\dLdMdNZaebebe\ejce\e ebebfdOdPdQZddRdSZedTdUZfejjgejjge eOdVdWdXZhe e eOe fe eOe fdYdZd[Zie d\d]d^d_Zje e eOe fd\d`dadbZkebebddcdddeZlejmfej`ej`ebebejcej`e\ejne ej`ej`fdf dgdhZoe ebd]didjZpejjge e dkfe eOe e dkffdldmdnZqdodpd d%d'd)d+d.d2d4d6d8d:dd@dBdDdGdIdKdNdQdUdnd0dhdegZrdS)qz? Utils shared by different modes of quantization (eager/graph) N) OrderedDict)getfullargspec signature)AnyCallableDictOptionalTupleUnion) QuantType)Node)is_parametrizedztorch.ao.quantization.utilsc@seZdZdZdS) MatchAllNodezo A node pattern that matches all nodes, used in defining fusion patterns in FX Graph Mode Quantization N)__name__ __module__ __qualname____doc__rrZ/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/torch/ao/quantization/utils.pyr"srreluZrelu_ contiguousdetachZdetach_ hardsigmoidZ hardsigmoid_Zpermuterepeatrepeat_interleaveZreshapeZresize_shapesigmoidZsigmoid_sizesqueezeZsqueeze_tanhZtanh_ transpose unsqueezeZ unsqueeze_viewcCsR|jdko|jtk}|jdko&|jtk}|jdkoFt|t|jtk}|||fS)NZ call_functionZ call_methodZ call_module)optarget func_list method_listtypestrmodule_type_list)nodemodulesZis_call_functionZis_call_methodZis_call_modulerrr check_nodets r,cCs|}|||SN)copyupdate)Z default_dictZadditional_dictdrrrget_combined_dict{s r1cCs|tjkp|tjkSr-)torchper_tensor_affineper_tensor_symmetricqschemerrr is_per_tensors r7cCs|tjtjtjfkSr-)r2per_channel_affine per_channel_affine_float_qparamsper_channel_symmetricr5rrris_per_channelsr;)objfqnreturncCstt|d|S)zO Given an obj and a fqn such as "foo.bar.baz", returns gm.foo.bar.baz. .) functoolsreducegetattrsplit)r<r=rrrgetattr_from_fqnsrDc CsHtjtjtjtjtjtjtjtjtjtji}||ks@t d|||S)NzUnsupported dtype: ) r2quint8uint8qint8Zint8qint32Zint32quint4x2Zquint2x4AssertionError)ZqdtypeZ DTYPE_MAPPINGrrrto_underlying_dtypesrKcCsddlm}t|dd}|j}||d}|r6t||r@d|dSt|rPtj}n2t|rt|tj krhtj }|j |d<nt d|||d<| \}}||d<||d<t|d r|j|d <t|d r|j|d <|S) Nr)PlaceholderObserverr6)r6dtypeZaxiszUnrecognized qscheme: scale zero_point quant_min quant_max)Ztorch.ao.quantization.observerrLrBrM isinstancer7r2r3r;r:r8Zch_axis RuntimeErrorZcalculate_qparamshasattrrPrQ)Zobserver_or_fake_quantrLr6rMZqparamsrNrOrrrget_qparam_dicts,           rUcCsDt|}||i}t||ks8tdt|d||t|S)a Get the observed/quantized custom module class that we need to swap `custom_module` to Input: custom_module: input, can be an instance of either a float or observed custom module custom_module_class_mapping: the float to observed or observed to quantized custom module class mapping qconfig: qconfig configured for the custom module Output: corresponding observed/quantized custom module class for input custom module instance z5did not find corresponding observed module class for z in mapping: )get_quant_typegetr'rJ)Z custom_moduleZcustom_module_class_mappingqconfigZ quant_typeZ class_mappingrrrget_swapped_custom_module_classs  $rYcCs|dk s t|}|jSr-)rJ activationrM)rXrZrrractivation_dtypes r[cCs|dk s t|}|jSr-)rJweightrM)rXr\rrr weight_dtypes r]cCs&t|tjtjtjtjfko$t| S)z Given a qconfig, decide if the activation needs to be quantized or not, this includes quantizing to quint8, qint8 and qint32 and float16 )r[r2rErGrHfloat16#activation_is_dynamically_quantizedrXrrr"activation_is_statically_quantizedsracCst|\}}}|S)z Given a qconfig, decide if the activation needs to be dynamically quantized or not, this includes dynamically quantizing to quint8, qint8 and float16 )get_qconfig_dtypes)rXr[_activation_is_dynamicrrrr_sr_cCst|tjtjfkS)z Given a qconfig, decide if the activation needs to be quantized to int8 or not, this includes quantizing to quint8, qint8 )r[r2rErGr`rrractivation_is_int8_quantizedsrecCst|tjkS)zY Given a qconfig, decide if the activation needs to be quantized to int32 or not )r[r2rHr`rrractivation_is_int32_quantizedsrfcCst|tjtjtjtjfkS)zL Given a qconfig, decide if the weight needs to be quantized or not )r]r2rErGr^rIr`rrrweight_is_quantizedsrgcCst|tjtjfkS)zW Given a qconfig, decide if the weight needs to be statically quantized or not )r]r2rErGr`rrrweight_is_statically_quantizedsrh)r>cCs&t|\}}}|tjko$|tjko$|S)zU Given a qconfig, returns True if this op is using int8 dynamic quantization )rbr2rErG)rXr[r]rdrrr op_is_int8_dynamically_quantizeds ricCs6|dk s t|}|}t|dd}|j|j|fS)zh returns the qconfig tuple for qconfig: (activation_dtype, weight_dtype, activation_is_dynamic) N is_dynamicF)rJrZr\rBrM)rXrZr\Zact_is_dynamicrrrrbs   rbcCs|dk s t|}|}tjtjtjtjg}|j|krft |drP|j rPt j S|j|kr`t j St jS|jtjkrt |dr|j rt j S|jtjkrt j Std|jd|jddS)Nrjz=Unrecognized dtype combination in get_quant_type: activation(z ),weight())rJrZr\r2rErGrIrHrMrTrjr ZDYNAMICZSTATICZ WEIGHT_ONLYr^ Exception)rXrZr\Z static_dtypesrrrrVs      rV)min_valmax_valr>cCs|dks|dkr&tddS|dks>|dkr|tdkrd|tdkrdtddS||kstd|d|n"t||kstd|d|dS) z Checks if the given minimum and maximum values are valid, meaning that they exist and the min value is less than the max value. rzMmust run observer before calling calculate_qparams. Returning default values.Finfz-infzmin z should be less than max T)ZnumelwarningswarndimfloatrJr2all)rmrnrrrcheck_min_max_valid0s$ru)rPrQhas_customized_qrangerM reduce_ranger>c Cs&|r|tjkrd\}}nd\}}||}}|dk rD|dk rD||}}||d} |tjkrzd| krndksntdn(|tjkrd| krdksntd |r|d |d }}nd|tjkr|rd \}}nd \}}nB|tjkr|rd \}}nd\}}n|tjkrd\}}nd\}}||fS)ztCalculates actual qmin and qmax based on the quantization range, observer datatype and if range is reduced. )rl)rNrzRquantization range should be positive and not exceed the maximum bit range (=256).lzYquantization range should be positive and not exceed the maximum bit range (=4294967296).)i?)i)rr})ii)r)r2rHrGrJrE) rPrQrvrMrwZinitial_quant_minZinitial_quant_maxZcustom_quant_minZcustom_quant_maxZ qrange_lenrrrcalculate_qmin_qmaxMsP                rcCs8|dd}t|dkr$d|dfS|d|dfSdS)z, Turn 'foo.bar' into ['foo', 'bar'] r?ryrN)rsplitlen)r$rrrr _parent_names   rcCs:t|jdkrdSt|r2t|jdko0d|jkSdSdS)z Checks if module._modules is empty or if module is a parametrization, checks that module._modules only has the 'parametrizations' module rTryZparametrizationsFN)rZ_modulesr )modulerrr)has_no_children_ignoring_parametrizationss r)root submoduler>cCs&|D]\}}||kr|SqdS)aS Get the path (fully qualified name) of a submodule Example:: >> class M(torch.nn.Module): def __init__(self): self.linear = torch.nn.Linear(5, 5) def forward(self, x): return self.linear(x) >> m = M() >> l = m.linear >> _get_path_of_module(m, l) "linear" N)Z named_modules)rrnprrr_get_path_of_modules r)flocr>csfdd|DS)z Get local keyword arguments Example:: >> def f(self, a, b=9): pass >> loc = {"a": 6, "c": 7} >> _get_signature_locals(f, loc) {"a": 6} cs$i|]\}}|tjkr||qSr)r parameters).0kvrrr sz)_get_signature_locals..)items)rrrrr_get_signature_localss rzOrderedDict[str, Any])rr>cCsfi}t|jD]J\}}|j|jk r2|j||<q|j|jkrHd||<q|j|jkri||<qt|S)z Get all default keyword arguments from function signature Example:: >> def f(self, a, b=9): pass >> _get_default_kwargs(f) {"b": 9} r) rrrdefaultemptykindVAR_POSITIONAL VAR_KEYWORDr)rkwargsnameparamrrr_get_default_kwargss       r)funcrr>cCs@t|}t||}|}|D]\}}||kr"|||<q"|S)a) Given a function and local function arguments, normalize the keyword arguments by filling in default arguments from function signature Example:: >> def f(self, key1=3, key2=3): pass >> loc = {"key2": 6} >> _normalize_kwargs(f, loc) {"key1": 3, "key2": 6} )rrr.r)rrZdefault_kwargsZ local_kwargsnormalized_kwargsattrvalrrr_normalize_kwargss   r)rPrQr>cCs2|dkr|ksntd||ks.tddS)aeValidates that the user-specified quantization range is properly initialized and within the given bound supported by the observer dtype. To accommodate lower-bit quantization with respect to the existing torch.qint8 and torch.quint8 datatypes, the user can choose to use dynamic quantization range by passing in a tuple of initial qmin and qmax values. One use case is these customized qmin and qmax values are used to calculate static estimates of the scale and zero point for aggressive lower-bit fake quantization. These estimates are compared against parameters learned through backpropagation. The related literatures for scale and zero point via backpropagation are as follows: Learned Step Size Quantization: https://openreview.net/pdf?id=rkgO66VKDS Trained Quantization Thresholds: https://arxiv.org/pdf/1903.08066.pdf rz1Used-specified quantization range must include 0.zKqmin must be strictly less than qmax for user-specified quantization range.N)rJ)rPrQrrrvalidate_qmin_qmaxs r) rmrnrPrQrMepsrvr6r>c Cs.t||s2tjdg|jjdtjdg|jjdfSt|t|}t|t|} |j} tj| tj | d} tj | tj | d} |tj ks|tjkrt| | } | t||d} t| |} |tjks|tjkr|r| | ||d} n| | d} n|tjkrZ||t||} t| |k| t| } d|| } nH| |t||} t| |} |t|| tj} t| ||} t| jdkrtjt| g| j| d} t| jdkrtjt| g| j| d} |tjkrtjt| g| j| d} | tj | tj fS)adCalculates the quantization parameters, given min and max value tensors. Works for both per tensor and per channel cases Args: min_val: Minimum values per channel max_val: Maximum values per channel Returns: scales: Scales tensor of shape (#channels,) zero_points: Zero points tensor of shape (#channels,) g?)devicer)rMrr{)rur2Ztensorrr'minZ zeros_likemaxZonesrdoubleZzerosZint64r4r:rsrFrEZnew_fullr9whereZ ones_likeroundtointclamprrrM) rmrnrPrQrMrrvr6Z min_val_negZ max_val_posrrNrOrrrdetermine_qparamssX (     rcCstt|jS)z Get number of positional args for a function Example:: >> def f(self, key1=3, key2=3): pass >> _get_num_pos_args(f) 3 )rrargsrrrr_get_num_pos_argsGs r.)modelexample_inputsr>csJ|ifdd}tjjj|tjj_z ||W5tjj_XS)a Given a model and its example inputs, return a dictionary from fully qualified name of submodules to example_inputs for that submodule, e.g. {"linear1": (tensor1,), "linear2": (tensor2,), "sub": (tensor3,), "sub.linear1": (tensor4,), ...} Used to make quantizing submodules easier now that FX Graph Mode Quantization requires example inputs. Also works for keyword arguments with default values, we would flatten keyword arguments as positional arguments and fill in the missing keyword args with default values, e.g. if we have a forward function: def forward(self, x, key1=3, key2=3): ... and we call it with self.submodule(x, key2=6) we'll get example_inputs: (x, 3, 6) user can also override `key1` with positional arguments as well: for self.submodule(x, 5, key2=6) we'll get: (x, 5, 6) variable positional arguments and variable positional keyword arguments in forward function are not supported currently, so please make sure no submodules is using them. c st|}t|j|}t|jd}|t|}|rP|rP|jdd|d8}q2||t |}t |}|dk r||<|f||S)NryF)last) listr.rforwardrrpopitemextendvaluestupler) selfrrZsubmodule_example_inputsrnum_argsZ num_to_popZsubmodule_example_inputs_tupler=Zfqn_to_example_inputsZorig_module_callrrr_patched_module_callss      z7get_fqn_to_example_inputs.._patched_module_call)r2nnModule__call__)rrrrrrget_fqn_to_example_inputsSs    r NodePatternPattern)srr@rp collectionsrinspectrrtypingrrrrr r r2Z torch.ao.quantization.quant_typer Ztorch.fxr Ztorch.nn.utils.parametrizer rrZ QuantizerClsrrrZReLUZReLU6ZAdaptiveAvgPool1dZAdaptiveAvgPool2dZAdaptiveAvgPool3dZ AvgPool1dZ AvgPool2dZ AvgPool3dZ MaxPool1dZ MaxPool2dZ MaxPool3dZIdentityZ HardsigmoidZSigmoidZTanhr)Z functionalZadaptive_avg_pool1dZadaptive_avg_pool2dZadaptive_avg_pool3dZeluZ hardswishZ instance_normZ layer_normZ leaky_reluZsiluZmishZdropoutZ max_pool1dZ max_pool2dZ max_pool3drZhardtanhZ hardtanh_rrr rrstacksumrr!catr%Zmeanr&r,r1r7r;r(rDrKrUrYr[r]rar_rerfrgrhboolrirbrVZTensorrurrMrrrrrrrrrr3r6rrr__all__rrrrsJ     &" $      5  "  G   :