U 0-e@sddlZddlZddlmZddlmZmZmZmZddl Z ddl Z ddl m Z dddZdd Zd d Zeeejfeee jfd d dZGdddeZGdddeZeee jfeedddZdS)N)Mapping)DictListOptionalUnion)is_safetensors_availablecCsd}t|jdkr"|tj}d}|}tj ||d}|dk rp|dkr\t|j}|t |j d||<|j dkr|d}t j||jd|j d}|dd|dd<||S)Nztorch.bfloat16bfloat16.dat)dtypeshaperzw+)r moder )strr viewtorchint16cpunumpyospathjoinlistr ndimnpmemmapflush)weight weight_nameoffload_folderindexr arrayZ tensor_fileZ file_arrayr!Y/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/accelerate/utils/offload.pyoffload_weights     r#cCs|t|d}|dkrd}|d}|dkr,d}tj|||dd}t|dd krV|d }t|}|ddkrx|tj}|S) Nr r!)rr r rr)r r r r)tuplerrlenrtensorrr ) weight_file weight_infor r rr!r!r"load_offloaded_weight/s    r*c Cs|dkst|dkrdStj|d}tj|rXt|ddd}t|}W5QRXni}||t|ddd}tj ||ddW5QRXdS) Nr index.jsonr$zutf-8)encodingw)indent) r&rrrisfileopenjsonloadupdatedump)rrZoffload_index_filefZ current_indexr!r!r"save_offload_indexEs  r7)save_dir state_dictcCsBtj|ddi}|D]\}}t||||d}qt||dS)a Offload a state dict in a given folder. Args: save_dir (`str` or `os.PathLike`): The directory in which to offload the state dict. state_dict (`Dict[str, torch.Tensor]`): The dictionary of tensors to offload. T)exist_ok)rN)rmakedirsitemsr#r7)r8r9rnameZ parameterr!r!r"offload_state_dictVs r>c@s8eZdZdZeedddZddZddZd d Z d S) PrefixedDatasetz Will access keys in a given dataset by adding a prefix. Args: dataset (`Mapping`): Any map with string keys. prefix (`str`): A prefix to add when trying to access any element in the underlying dataset. datasetprefixcCs||_||_dSNr@)selfrArBr!r!r"__init__rszPrefixedDataset.__init__cCs|j|j|SrCr@)rDkeyr!r!r" __getitem__vszPrefixedDataset.__getitem__cstfddjDS)Ncsg|]}|jr|qSr!) startswithrB.0rFrDr!r" zs z,PrefixedDataset.__iter__..)iterrArKr!rKr"__iter__yszPrefixedDataset.__iter__cCs t|jSrC)r&rArKr!r!r"__len__|szPrefixedDataset.__len__N) __name__ __module__ __qualname____doc__rrrErGrNrOr!r!r!r"r?is r?c@sZeZdZdZd eeejfee ee j fe dddZ edddZd d Zd d ZdS)OffloadedWeightsLoaderaC A collection that loads weights stored in a given state dict or memory-mapped on disk. Args: state_dict (`Dict[str, torch.Tensor]`, *optional*): A dictionary parameter name to tensor. save_folder (`str` or `os.PathLike`, *optional*): The directory in which the weights are stored (by `offload_state_dict` for instance). index (`Dict`, *optional*): A dictionary from weight name to their information (`dtype`/ `shape` or safetensors filename). Will default to the index saved in `save_folder`. N)r9 save_folderrc s|dkr |dkr |dkr td|dkr,in|_|_|dkrp|dk rpttj|d}t|}W5QRX|dkr|in|_ t j _ j fddj D|_dS)NzWNeed either a `state_dict`, a `save_folder` or an `index` containing offloaded weights.r+csg|]}|jkr|qSr!)all_keysrIrKr!r"rLs z3OffloadedWeightsLoader.__init__..) ValueErrorr9rUr1rrrr2r3rrkeysrVextenddevice)rDr9rUrrZr6r!rKr"rEszOffloadedWeightsLoader.__init__)rFc Cs||jkr|j|S|j|}|ddk rts:tdddlm}|jdkrTdn|j}||dd|d}||d|}W5QRXd |kr| t t |d S|St j |j|d }t||S) NZsafetensors_filezRThese offloaded weights require the use of safetensors: `pip install safetensors`.r) safe_openrpt)Z frameworkrZrr r )r9rgetr ImportErrorZ safetensorsr[rZZ get_tensortogetattrrrrrrUr*)rDrFr)r[rZr6r'r(r!r!r"rGs    z"OffloadedWeightsLoader.__getitem__cCs t|jSrC)rMrVrKr!r!r"rNszOffloadedWeightsLoader.__iter__cCs t|jSrC)r&rVrKr!r!r"rOszOffloadedWeightsLoader.__len__)NNNN)rPrQrRrSrrrTensorrrrPathLikerrErGrNrOr!r!r!r"rTs  rT)r9submodule_namescs.i}|D] |fdd|Dq|S)a Extract the sub state-dict corresponding to a list of given submodules. Args: state_dict (`Dict[str, torch.Tensor]`): The state dict to extract from. submodule_names (`List[str]`): The list of submodule names we want to extract. cs,i|]$\}}|ks"|dr||qS).)rH)rJrFparam module_namer!r" s z1extract_submodules_state_dict..)r4r<)r9rcresultr!rfr"extract_submodules_state_dicts rj)N)r2rcollections.abcrtypingrrrrrrrZimportsrr#r*r7rrbrar>r?rTrjr!r!r!r"s   $@