U +-eA@sddlZddlZddlmZddlmZmZmZmZm Z m Z m Z ddl Z ddl mZmZmZmZe jedddZe jeddd Ze jedd d Zee eeee jfee ed d dZeee jfee edddZe jedddZdddeee jfeeeeeeeeeefdddZd=e jjeeeeefedddZd>e jje eej fe eeeefdddZ!d?eee jfeeeefe"dd d!Z#d@eee jfe eej feeeefd"d#d$Z$dAe eej feee jfd&d'd(Z%e"eee jfd)d*d+Z&e j'd,e j(d-e j)d-e j*d.e j+d.e j,d.e j-d/e j.d/e jd/e j/d,i Z0e j/e j(e j+e j*e j'e j)e j,e j.e j-e jd0 Z1ee j2d1d2d3Z3eee jfd4d5d6Z4e jee"d7d8d9Z5eee jfeeeeeffd:d;d<Z6dS)BN) defaultdict)AnyDictListOptionalSetTupleUnion) deserialize safe_open serializeserialize_file)tensorreturnc CsTz|WStk rNz|WYStk rHYYdSXYnXdS)Nr)untyped_storagedata_ptr ExceptionstorageNotImplementedErrorrrR/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/safetensors/torch.py storage_ptr srcCs2|r&|ddt|j}n|}|S)N)nelementviewr_SIZEdtype)rstoprrr_end_ptrsrc Cspz|WStk rjz|t|jWYStk rd|t|jYYSXYnXdSN) rnbytesAttributeErrorrsizerrrrrrrr storage_sizesr$)tensors state_dictrc Csg}|D]}t|dkr$||qg}|D]$}||}||t||fq,||d\}}} || h|ddD]2\} } }| |kr||hn|d|| }qq|S)Nrr)lenappendrrsortadd) r%r&Zfiltered_tensorssharedZareasnamer_Z last_stop last_namestartrrrr_filter_shared_not_shared,s$   r2)r&rcCs~tt}|D]N\}}|jtdkrt|dkrt|dkr||jt|t|f|qtt | }t ||}|S)Nmetar) rsetitemsdevicetorchrr$r,listsortedvaluesr2)r&r%kvrrr_find_shared_tensorsEs(  r=cCs*|t|ko(|t|jt|kSr )rrrrrr$rrrr _is_completePsr>)preferred_names discard_names)r&r?r@rc s|dkr g}t|}|dkr g}t|}t}tt}|D]}tfdd|D}|sjtd|dtt|d}||}|rtt|d}|r||}|rtt|d}t|D]} | |kr||| qq<|S)Ncsg|]}t|r|qSr)r>).0r.r&rr ds z+_remove_duplicate_names..zvError while trying to find names to remove to save state dict, but found no suitable name to keep for saving amongst: z. None is covering the entire storage.Refusing to save/load the model since you could be storing much more memory than needed. Please refer to https://huggingface.co/docs/safetensors/torch_shared_tensors for more information. Or open an issue.r) r4r=rr8 RuntimeErrorr9 difference intersectionr*) r&r?r@Zshareds to_remover-Zcomplete_namesZ keep_name preferredr.rrBr_remove_duplicate_namesTs4    rIT)modelfilenamemetadataforce_contiguousc Cs|}t|}|D]4\}}|D]&}|dkr4i}||krD|||<||=q$q|rddd|D}zt|||dWn:tk r} zt| } | d7} t| W5d} ~ XYnXdS)a Saves a given torch model to specified filename. This method exists specifically to avoid tensor sharing issues which are not allowed in `safetensors`. [More information on tensor sharing](../torch_shared_tensors) Args: model (`torch.nn.Module`): The model to save on disk. filename (`str`): The filename location to save the file metadata (`Dict[str, str]`, *optional*): Extra information to save along with the file. Some metadata will be added for each dropped tensors. This information will not be enough to recover the entire shared structure but might help understanding things force_contiguous (`boolean`, *optional*, defaults to True): Forcing the state_dict to be saved as contiguous tensors. This has no effect on the correctness of the model, but it could potentially change performance if the layout of the tensor was chosen specifically for that reason. NcSsi|]\}}||qSr) contiguousrAr;r<rrr szsave_model..rLzT Or use save_model(..., force_contiguous=True), read the docs for potential caveats.)r&rIr5 save_file ValueErrorstr) rJrKrLrMr& to_removesZ kept_nameto_remove_grouprGemsgrrr save_models" rY)rJrKrc Cst|}|}t||d}|j|dd\}}t|}|D],}|D]"} | |krb|| qJ|| qJqB|r|s||rd ddt |D} d ddt |D} d|j j d } |r| d | 7} |r| d | 7} t | ||fS) an Loads a given filename onto a torch model. This method exists specifically to avoid tensor sharing issues which are not allowed in `safetensors`. [More information on tensor sharing](../torch_shared_tensors) Args: model (`torch.nn.Module`): The model to load onto. filename (`str`, or `os.PathLike`): The filename location to load the file from. strict (`bool`, *optional*, defaults to True): Wether to fail if you're missing keys or having unexpected ones When false, the function simply returns missing and unexpected names. Returns: `(missing, unexpected): (List[str], List[str])` `missing` are names in the model which were not modified during loading `unexpected` are names that are on the file, but weren't used during the load. )r?F)strictz, cSsg|]}d|dqS"rrAr;rrrrCszload_model..cSsg|]}d|dqSr[rr]rrrrCsz#Error(s) in loading state_dict for :z# Missing key(s) in state_dict: z& Unexpected key(s) in state_dict: ) load_filer&rIkeysZload_state_dictr4r:r*removejoinr9 __class____name__rD) rJrKrZr&Zmodel_state_dictrUmissingZ unexpectedrVrGZ missing_keysZunexpected_keyserrorrrr load_models(   rg)r%rLrcCstt||d}t|}|S)a$ Saves a dictionary of tensors into raw bytes in safetensors format. Args: tensors (`Dict[str, torch.Tensor]`): The incoming tensors. Tensors need to be contiguous and dense. metadata (`Dict[str, str]`, *optional*, defaults to `None`): Optional text only metadata you might want to save in your header. For instance it can be useful to specify more about the underlying tensors. This is purely informative and does not affect tensor loading. Returns: `bytes`: The raw bytes representing the format Example: ```python from safetensors.torch import save import torch tensors = {"embedding": torch.zeros((512, 1024)), "attention": torch.zeros((256, 256))} byte_data = save(tensors) ``` rQ)r _flattenbytes)r%rLZ serializedresultrrrsavesrkr%rKrLcCstt|||ddS)ah Saves a dictionary of tensors into raw bytes in safetensors format. Args: tensors (`Dict[str, torch.Tensor]`): The incoming tensors. Tensors need to be contiguous and dense. filename (`str`, or `os.PathLike`)): The filename we're saving into. metadata (`Dict[str, str]`, *optional*, defaults to `None`): Optional text only metadata you might want to save in your header. For instance it can be useful to specify more about the underlying tensors. This is purely informative and does not affect tensor loading. Returns: `None` Example: ```python from safetensors.torch import save_file import torch tensors = {"embedding": torch.zeros((512, 1024)), "attention": torch.zeros((256, 256))} save_file(tensors, "model.safetensors") ``` rQN)r rhrlrrrrRsrRcpu)rKrc Cs>i}t|d|d"}|D]}||||<qW5QRX|S)a Loads a safetensors file into torch format. Args: filename (`str`, or `os.PathLike`): The name of the file which contains the tensors device (`Dict[str, any]`, *optional*, defaults to `cpu`): The device where the tensors need to be located after load. available options are all regular torch device locations Returns: `Dict[str, torch.Tensor]`: dictionary that contains name as key, value as `torch.Tensor` Example: ```python from safetensors.torch import load_file file_path = "./my_folder/bert.safetensors" loaded = load_file(file_path) ``` pt)Z frameworkr6)r r`Z get_tensor)rKr6rjfr;rrrr_s  r_)datarcCst|}t|S)a Loads a safetensors file into torch format from pure bytes. Args: data (`bytes`): The content of a safetensors file Returns: `Dict[str, torch.Tensor]`: dictionary that contains name as key, value as `torch.Tensor` on cpu Example: ```python from safetensors.torch import load file_path = "./my_folder/bert.safetensors" with open(file_path, "rb") as f: data = f.read() loaded = load(data) ``` )r _view2torch)rpZflatrrrload:srrr'r() ZF64ZF32ZF16ZBF16ZI64ZI32ZI16ZI8ZU8ZBOOL) dtype_strrcCst|Sr )_TYPES)rurrr _getdtypessrw)rcCsfi}|D]X\}}t|d}tj|d|d|d}tjdkrXt|jdd}|||<q|S)Nrrp)rshapebigFZinplace) rwr7Z frombufferZreshapesys byteorderZ from_numpynumpybyteswap)Zsafeviewrjr;r<rZarrrrrrqws    rq)rr.rc Cs:|jtjkrtd|d|s4td|d|jjdkrJ|d}ddl}ddl }t | |j  }t|j}||}|}|dkrdS||||j}|j||f} tjdkr2tj|jtj|jtj|jtj|jtj|jtj|jtj|jtj|jtj t tj!|j!i } | |j} | "| j#d d } | $S) Nz)You are trying to save a sparse tensor: `` which this library does not support. You can make it a dense tensor before saving with `.to_dense()` but be aware this might make a much larger file than needed.z1You are trying to save a non contiguous tensor: `a` which is not allowed. It either means you are trying to save tensors which are reference of each other in which case it's recommended to save only the full tensors, and reslice at load time, or simply call `.contiguous()` on your tensor to pack it before saving.rmrryFrz)%layoutr7stridedrSZ is_contiguousr6typetoctypesr}intprodrxitemrrrcastZPOINTERZc_ubyteZ ctypeslibZas_arrayr{r|int64float32int32bfloat16float16int16uint8int8boolfloat64rr~tobytes) rr.rnplengthZbytes_per_item total_bytesZptrZnewptrrpZNPDTYPESZnpdtyperrr_tobytessZ        r)r%rcCst|tstdt|g}|D]B\}}t|tjsTtd|dt||jtjkr(| |q(|rtd|dt |}g}|D]}t |dkr| |q|rt d|dd d |DS) Nz4Expected a dict of [str, torch.Tensor] but received zKey `z1` is invalid, expected torch.Tensor but received z*You are trying to save a sparse tensors: `rr(z Some tensors share memory, this will lead to duplicate memory on disk and potential differences when loading them again: z. A potential way to correctly save your model is to use `save_model`. More information at https://huggingface.co/docs/safetensors/torch_shared_tensors cSs6i|].\}}|t|jdd|jt||dqS).r)rrxrp)rTrsplitrxrrOrrrrPs z_flatten..) isinstancedictrSrr5r7Tensorrrr*r=r)rD)r%Zinvalid_tensorsr;r<Zshared_pointersZfailingnamesrrrrhs4       rh)NT)T)N)N)rm)7osr{ collectionsrtypingrrrrrrr r7Z safetensorsr r r r rrrrr$rTr2r=rr>rInnModulerYPathLikergrirkrRr_rrrrrrrrrrrrrvrrwrqrrhrrrrs $  ,"     / .2,*!   "& 7