U 0-e@sddlZddlmZmZmZddlZddlZddlm Z ddl m Z ddl m Z mZddlmZmZmZed d rddlmmZdeed d d ZdeeeejdddZdeeeefeejdddZdS)N)ListOptionalUnion)AcceleratorState)CUDA_DISTRIBUTED_TYPES)DistributedTypeRNGType)is_npu_availableis_tpu_availableis_xpu_availableF)Z check_deviceseedZdevice_specificcCsx|r|tj7}t|tj|t|trDtj |n t rXtj |n tj |t rtt|dS)a= Helper function for reproducible behavior to set the seed in `random`, `numpy`, `torch`. Args: seed (`int`): The seed to set. device_specific (`bool`, *optional*, defaults to `False`): Whether to differ the seed on each device slightly with `self.process_index`. N)rZ process_indexrandomrnptorchZ manual_seedr xpuZmanual_seed_allr npucudar xm set_rng_staterrX/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/accelerate/utils/random.pyset_seeds      r)rng_type generatorcCs|tjkrt}n|tjkr*tj}n|tjkrRtsBtdt t }nj|tj krvt sjtdtj }nF|tjkrtstdtj}n"|tjkr|dk std|}t}|jtjkr|t }t |gt |}nh|jtks"|jtjks"|jtjkrF||j}tj !|d|}n|jtj"krbtj !|d|tjkrzt#|n|tjkrtj#|nf|tj krtj #|nL|tjkrtj#|n2|tjkrt #|$n|tjkr|%|dS)Nz;Can't synchronize XLA seeds on an environment without TPUs.z;Can't synchronize NPU seeds on an environment without NPUs.z;Can't synchronize XPU seeds on an environment without XPUs.z)Need a generator to synchronize its seed.r)&r ZTORCHrZ get_rng_stateCUDArZXLAr AssertionErrorZtensorrZNPUr rZXPUr r GENERATORZ get_staterZdistributed_typer ZTPUtoZ xla_deviceZcollective_broadcastZ mark_stepcpurZ MULTI_NPUZ MULTI_XPUZdevice distributed broadcastZ MULTI_CPUritemZ set_state)rrZ rng_statestaterrrsynchronize_rng_state9sZ                        r&) rng_typesrcCs|D]}tt||dqdS)N)r)r&r )r'rrrrrsynchronize_rng_statesmsr()F)NN)N)rtypingrrrnumpyrrr%r constantsr dataclassesr r Zimportsr r r Ztorch_xla.core.xla_modelcoreZ xla_modelrintboolr Generatorr&strr(rrrrs   4