U ,-e@sddlmZmZmZmZddlmZmZddlm Z ddl m Z ddddd d gZ e j ee j ed d dZe j eee j ee j fe j d ddZde j e j eeedddZee j eee j dddZe j eeeefddd ZGdd d e j ZdS))AnyDictOptionalType)type_before_parametrizationsis_parametrized)chain)nnmodule_contains_param swap_module module_to_fqn fqn_to_moduleget_arg_info_from_tensor_fqn FakeSparsity)moduleparametrizationreturncs(t|r$tfdd|jDSdS)Nc3s(|] \}}tfdd|DVqdS)c3s|]}t|VqdSN) isinstance).0paramrb/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/torch/ao/pruning/sparsifier/utils.py sz2module_contains_param...N)any)rkeyZ param_listrrrrsz(module_contains_param..F)rrZparametrizationsitems)rrrrrr s  )modmappingrcCst||kr|t|}||}|jD]}||q,|jD]}||qFddt|| D}t |dkst d|t |dkrt t |nd}|r|||S|SdS)a%Swaps the module using from_dense according to the mapping passed in. Args: mod: input module mapping: a dictionary that maps from nn module to sparse nn module Return: The corresponding sparse module of `mod` according to mapping, created using from_dense cSsh|] }|jqSr)device)rprrr 4szswap_module..zOswap_module only works with cpu or single-device CUDA modules, but got devices rN)rZ from_denseZ_forward_pre_hooksvaluesZregister_forward_pre_hookZ_forward_hooksZregister_forward_hookr parametersbufferslenAssertionErrornextiterto)rrZ sparse_modnew_modZ pre_hook_fnZhook_fnZdevicesr rrrr s      )modelrprefixrcCsH||kr dS|D].\}}t||d}t|tr|||SqdS)zS Returns the fqn for a module or None if module not a descendent of model. r-.N)Znamed_childrenr rstr)r.rr/namechildZfqnrrrr Bs  )r.pathrcCs(|dkr$|dD]}t||d}q|S)z Given an fqn, returns the corresponding module or tensor or None if the fqn given by `path` doesn't correspond to anything. Similar to model.get_submodule(path) but works for tensors. r-r0N)splitgetattr)r.r4r2rrrr Qs)r. tensor_fqnrcCs@|dd}|dt| d|k}t||}||||dS)zX Uses tensor_fqn to obtain a dict containing module_fqn, module and tensor_name r0N) module_fqnr tensor_namer7)r5r'r )r.r7r:r9rrrrr\s cs0eZdZdZfddZddZddZZS)ra;Parametrization for the weights. Should be attached to the 'weight' or any other parameter that requires a mask applied to it. Note:: Once the mask is passed, the variable should not change the id. The contents of the mask can change, but the mask reference itself should not. cst|d|dS)Nmask)super__init__Zregister_buffer)selfr; __class__rrr=|s zFakeSparsity.__init__cCs|jj|jkst|j|Sr)r;shaper()r>xrrrforwardszFakeSparsity.forwardcOsiSrr)r>argskwargsrrr state_dictszFakeSparsity.state_dict)__name__ __module__ __qualname____doc__r=rCrF __classcell__rrr?rrqs N)r-)typingrrrrZtorch.nn.utils.parametrizerr itertoolsrZtorchr __all__Moduleboolr r r1r r rrrrrrs2     (