U ,-e 9 @s2ddlZddlmZddlmZddlmZddddd d d d d g ZddZddZ ddZ ddZ d/ddZ dej fddZddej fddZd0ddZd1dd Zd2ddZddej dfdd Zddej fd!d"Zd3d#d Zd4d$d Zd5d%d&Zd6d'd(Zddddej fd)d*Zd7d+d Zd8d,d Zd9d-d.ZdS):N)QConfig) QuantType)wrap_cpp_modulescript_qconfigscript_qconfig_dictfuse_conv_bn_jit prepare_jitprepare_dynamic_jit convert_jitconvert_dynamic_jit quantize_jitquantize_dynamic_jitcCs&t|tjjs"tdtt|dS)Nz$input must be a script module, got: ) isinstancetorchjitZ ScriptModule ValueErrorstrtypemodelrc/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/torch/ao/quantization/quantize_jit.py_check_is_script_modulesrcCs|jdstddS)Nforwardz0input script module does not have forward method)_cZ _has_methodrrrrr_check_forward_methods rcCs(ttj|jtj|jdS)zInstantiate the activation and weight observer modules and script them, these observer module instances will be deepcopied during prepare_jit step. ) activationweight)rrrscriptrrr)ZqconfigrrrrscCsdd|DS)z~Helper function used by `prepare_jit`. Apply `script_qconfig` for all entries in `qconfig_dict` that is not None. cSs"i|]\}}||rt|ndqSN)r).0kvrrr )sz'script_qconfig_dict..)items) qconfig_dictrrrr$sFcCs:tjd|j}tj|}|r.||nt|}|S)z Fuse conv - bn module Works for eval model only. Args: model: TorchScript model from scripting or tracing z.quantization_api.quantize_jit.fuse_conv_bn_jit)r_C_log_api_usage_oncerZ_jit_pass_fold_convbn _reconstructr)rinplacemodel_crrrr+s   cCsrt|t|tdd|Ds.tdt|}t||}tj |j d|||}|rf| |nt |}|S)Ncss|]}t|tVqdSrrrr xrrr >sz_prepare_jit..4qconfig_dict should only contain names(str) as keys.r) rrallkeysrrrrr&Z_jit_pass_insert_observersrr(r)rr%r) quant_typescripted_qconfig_dictr*rrr _prepare_jit;s    r4rcCst|tdd|Ds&tdt|}|j|j}tj |t ||}tj |j||||}|rx| |nt|}|S)Ncss|]}t|tVqdSrr+r,rrrr.Osz(_prepare_ondevice_jit..r/)rr0r1rrrZ _get_methodgraphrr&Z_jit_pass_inlinerZ1_jit_pass_insert_observer_method_for_ondevice_ptqr(r)rr% method_namer)r2r3Z method_graphr*rrr_prepare_ondevice_jitMs"    r7cCstjdt|||tjdS)Nz)quantization_api.quantize_jit.prepare_jitr2)rr&r'r4rSTATICrr%r)rrrr`s cCstjdt|||tjdS)Nz1quantization_api.quantize_jit.prepare_dynamic_jitr8)rr&r'r4rDYNAMICr:rrrr ds cCst||||tjdSNr8)r7rr;rr%r6r)rrr_prepare_ondevice_dynamic_jitisr>cCst|||j}tj|d|||}|sltdd|D}|sP||dkr\g}tj |||}|r|| |nt |}tj |j tj|j |S)Nrcss|]}|jjdkVqdS)ZxpuN)Zdevicer)r prrrr.ssz_convert_jit..)revalrrr&Z_jit_pass_insert_quant_dequantr0 parameterscpuZ_jit_pass_quant_finalizer(r_jit_pass_constant_propagationr5 _jit_pass_dce)rr)debugr2preserved_attrsr*Zis_xpurrr _convert_jitls" rGcCst||tjkstd|dr,tdd|}d|}|j}tj|j|||tj}tj |tj|}|r|| |nt |}|S)NzOThis API, while should work for static quant, is only tested for dynamic quant.Zobserve_z2Pass in valid method to be quantized, e.g. forwardZ quantize_) rrr;AssertionError startswithrrr&Z/_jit_pass_insert_quant_dequant_for_ondevice_ptqZ)_jit_pass_quant_finalize_for_ondevice_ptqr(r)rr6r)rEr2Zobserve_method_nameZquantize_method_namer*rrr_convert_ondevice_jits$ rJcCs tjdt|||tj|dS)Nz)quantization_api.quantize_jit.convert_jitr2rF)rr&r'rGrr9rr)rErFrrrr s cCs tjdt|||tj|dS)Nz1quantization_api.quantize_jit.convert_dynamic_jitrK)rr&r'rGrr;rLrrrr s cCst||||tjdSr<)rJrr;)rr6r)rErrr_convert_ondevice_dynamic_jitsrMcCst||||}t|||}|Sr)r>rMr=rrr#_quantize_ondevice_dynamic_jit_impls rNcCs|tjkr$t|||}t|d|}n>|s0td|s is added that observe the values to be quantized. - reset_observers_ to reset observers. - quantize_ is added to the model. - This method extract scale, zero points. - Quantizes observed weights. - Creates packed params from it and update the attribute of the model with the new values for the packed params. - Reset the original fp32 weights with empty tensor using SetAttr. - quantized_ is added to the model. - This method uses quantized weights and quantized linear ops instead of fp32 op. - This method should be used for inference post PTQ. - Note that all method's signatures should be the same as method_name. Later on device: - Run reset_observers_ - Run observe_ - Run quantize_ - Now model can be saved and loaded later. - Run model with quantized_ Example: ```python import torch from torch.ao.quantization import per_channel_dynamic_qconfig from torch.ao.quantization.quantize_jit import _quantize_ondevice_dynamic_jit ts_model = torch.jit.script(float_model.eval()) # or torch.jit.trace(float_model, input) qconfig = get_default_qconfig('fbgemm') quant_ready_model = _quantize_ondevice_dynamic_jit( ts_model, {'': qconfig}, 'forward', True) ``` )r))rNr=rrr_quantize_ondevice_dynamic_jits9rR)F)F)F)rF)FFN)FFN)FF)F)FF)FF)rF)rZtorch.ao.quantization.qconfigrZ torch.ao.quantization.quant_typerZtorch.jit._recursiver__all__rrrrrr9r4r7rr r>rGrJr r rMrNrQr r rRrrrrsF               5 *