U 9%e@sddlZddlmZmZddlZddlmZddlmZddl m Z ddl m Z dd l mZGd d d e Zd d ZddZGdddeZGdddeZGdddeZGdddeZdS)N)DictList) variables) unimplemented)istype)VariableTrackerConstantVariablecs(eZdZfddZeddZZS)DistributedVariablec s"tjf|tstddS)Nz+torch.distributed package is not available!)super__init__r is_availabler)selfkwargs __class__b/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/torch/_dynamo/variables/distributed.pyr szDistributedVariable.__init__cCs tjSN)torch distributedrrrrrrsz DistributedVariable.is_available)__name__ __module__ __qualname__r staticmethodr __classcell__rrrrr s r cCs,ts dSddlm}t|o*||jkS)NFr)DTensor)r rZtorch.distributed._tensorrinspect isfunctionZ from_local)valuerrrr is_from_locals r"cCs6ts dSddlm}m}||g}t|o4||kS)NFr)_get_group_tagget_process_group_ranks)r rZ"torch.distributed.distributed_c10dr#r$rr )r!r#r$Zconstant_processgroup_functionsrrris_constant_pg_functions sr%cs>eZdZfddZeddZddddfd d ZZS) PlacementClassVariablec stjf|||_dSrr rr!rr!rrrrr2szPlacementClassVariable.__init__cCs.ts dSddlm}t|tko,t||SNFr) Placement)r r)torch.distributed._tensor.placement_typesr*type issubclassr!r*rrris_placement_type6s z(PlacementClassVariable.is_placement_typeList[VariableTracker]Dict[str, VariableTracker]r argsrreturncs~t|||}t|jddtjfkrn|jrnt|j}t |f|}t|jddrn| | |d||St |||S)N__new__r)r propagatevaluesrgetattr_staticr!objectr5sourcePlacementVariable add_options call_methodr call_function)rtxr3roptionsZnew_objvarrrrr>@s  z$PlacementClassVariable.call_function)rrrrrr/r>rrrrrr&1s  r&csFeZdZfddZeddZddZddd d fd d ZZS) r;c stjf|||_dSrr'r(rrrrSszPlacementVariable.__init__cCs"ts dSddlm}t||Sr))r rr+r*rr.rrr is_placementWs zPlacementVariable.is_placementcCs|jSrr!rrrras_python_constantasz$PlacementVariable.as_python_constantr0r1r r2c sddlm}t|||}ddg}||krz4t|j}t|dddksTt dt||} Wnt k rzd} YnX| t j kr|d |Sdd|D}d d | D}| |jf|||St||||S) Nrr r __setattr__ __getattr__zno custom getattr allowed!cSsg|] }|qSrrE).0xrrr }sz1PlacementVariable.call_method..cSsi|]\}}||qSrrH)rIkvrrr ~sz1PlacementVariable.call_method..)N)r r r6r7r,r!rr8AssertionErrorAttributeErrorr9ritemsr r=) rr?namer3rr r@allowed_methodsZ value_typemethodrrrr=ds(     zPlacementVariable.call_method) rrrrrrBrEr=rrrrrr;Rs  r;csDeZdZfddZeddZddZeedfdd Z Z S) DeviceMeshVariablec stjf|||_dSrr'r(rrrrszDeviceMeshVariable.__init__cCs"ts dSddlm}t||S)NFr) DeviceMesh)r rZ%torch.distributed._tensor.device_meshrWr)r!rWrrris_device_meshs z!DeviceMeshVariable.is_device_meshcCs|jSrrCrDrrrrEsz%DeviceMeshVariable.as_python_constant)rSr4cs"|dkrt|jjSt||S)Nndim)r r!rYr var_getattrrr?rSrrrrZs zDeviceMeshVariable.var_getattr) rrrrrrXrEstrr rZrrrrrrVs   rVcs^eZdZdZfddZddZddZdd d d fd d ZfddZe ddZ Z S)ProcessGroupVariablea We don't want a ProcessGroup object to end up in our output graph. But it's common for dynamo to intercept a PG that is then used to get info like rank() or world_size(), as well as passed to utility functions in distributed_c10d which desugar it into plain types like a ranklist and tag. For convenience and proper guarding, we construct a variable type. TODO: make it possible to use ProcessGroupVariable as input to simple functions like _expand_group without dynamo complaining about making a proxy for it. It is not a tensor-like type, and we don't want a proxy- but dynamo assumes torch library functions are dealing with tensor-like types and would have proxies for their args. TODO: should we make this inherit VT instead of UDOV? Do we want any of the default behaviors or just graph-break whenever one of our special cases is not hit? c stjf|||_dSrr'r(rrrrszProcessGroupVariable.__init__cCs|jSrrCrDrrrrEsz'ProcessGroupVariable.as_python_constantcCs t|jSr)r,r!rDrrr python_typesz ProcessGroupVariable.python_typer0r1r r2csB|dkrt|jS|dkr0t|jSt||||S)Nranksize)rr r!r_r`r r=)rr?rSr3rrrrr=s z ProcessGroupVariable.call_methodcs2dkr$tfddStS)N)r_r`cs||Sr)r=)r3rrSrr?rrz2ProcessGroupVariable.var_getattr..)rZLambdaVariabler<r rZr[rrarrZsz ProcessGroupVariable.var_getattrcCs"ts dSddlm}t||S)NFr) ProcessGroup)r rZtorch._C._distributed_c10drdr)r!rdrrris_process_groups z%ProcessGroupVariable.is_process_group) rrr__doc__rrEr^r=rZrrerrrrrr]s  r])rtypingrrrrOrexcrutilsrbaser Zconstantr r r"r%r&r;rVr]rrrrs      !3