U ×9%eÝÐã@sÎdZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZmZmZmZmZddlmZddlmZddlZddlZddlmZmZmZmZmZmZmZdd l m!Z!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*dd l+m,Z,d d l-m.Z.m/Z/d dl0m1Z1d dl2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8d dl/m9Z9e/ :e;¡Zdd¡ ?¡e3krrdndZ@dd„ZAe BdejC De Bdd¡d¡¡ZEejC DeEd¡ZFejC Ge BdejC De Bdd¡d¡¡¡ZHejC DeHd¡ZIejC JeF¡r4ejC JeI¡s4dej=kr4d ej=kr4d!ej=kr4e< Kd"¡e LeFeI¡e BdeI¡ZMe Bd eM¡ZNe Bd#eN¡ZOe Bd!eO¡ZPe Bd$ejC DeHd%¡¡ZQd&ZReƒjSZTe Bd'd¡e3kZUd(ZVd)ZWej= >d*d+¡ ?¡e3kZXeXr¾d,nd-ZYeYZZej= >d.d¡dk rôe [d/e\¡ej= >d.d¡ZZej= >d0eZ¡ZZeZd1Z]eZd2Z^e_ƒZ`d3d4„Zadreebe feed5œd6d7„Zcd8d9„Zddseeebdfebd:œd;d<„Zeeebeebd=œd>d?„Zfdtebebeebe dfeebeebeebd@œdAdB„ZgdueebejhfebeeebejhfeieieeebebfeeeiebfeebeiebeebeeebeebebffeieieebdDœdEdF„ZjdveebejhfebeeebejhfeieieeebebfeeeiebfeebeiebdGœ dHdI„ZkdwdJdK„ZldxeebejhfebeebeeebebfeeeiebfdLœdMdN„ZmGdOdP„dPƒZndQdRœdSdT„ZoeepebfdUœdVdW„ZqdydXdY„ZrdzdZd[„Zsd\d]„Ztd^d_„Zud`da„Zvd{dbdc„ZwGddde„deƒZxejC DePdf¡ZyejC zey¡s\dZ{n@e|eyƒ2Z}zepe} ~¡ƒZ{Wnek rdZ{YnXW5QRXejC JeP¡oºe€e eP¡ƒdkZ‚e{d krde‚rdeAƒràe< Kdg¡n e< Kdh¡z ePeIkrewePePƒnewƒWnXeƒk rbZ„z8di De …e„j†¡¡Z‡e< ˆdje‡›die„j‰j;›dke„›dl¡W5dZ„[„XYnXe{d krÊz2ejŠePddme|eydnƒZ}e} ‹do¡W5QRXWn(eƒk rÈe< KdpeP›dq¡YnXdS)|z? Hub utilities: utilities related to download and cache models éN)Úfutures)ÚPath)ÚDictÚListÚOptionalÚTupleÚUnion)Úurlparse)Úuuid4)ÚCommitOperationAddÚ create_branchÚ create_commitÚ create_repoÚget_hf_file_metadataÚhf_hub_downloadÚ hf_hub_url)ÚREGEX_COMMIT_HASHÚhttp_get)ÚEntryNotFoundErrorÚGatedRepoErrorÚLocalEntryNotFoundErrorÚRepositoryNotFoundErrorÚRevisionNotFoundErrorÚbuild_hf_headersÚhf_raise_for_status)Ú HTTPErroré)Ú __version__Úlogging)Úworking_or_temp_dir)ÚENV_VARS_TRUE_VALUESÚ _tf_versionÚ_torch_versionÚis_tf_availableÚis_torch_availableÚis_training_run_on_sagemaker)ÚtqdmZTRANSFORMERS_OFFLINEÚ0TFcCstS©N)Ú_is_offline_mode©r*r*úU/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/transformers/utils/hub.pyÚis_offline_modeHsr,Z TORCH_HOMEÚXDG_CACHE_HOMEz~/.cacheZtorchÚ transformersZHF_HOMEZ huggingfaceZhubÚPYTORCH_PRETRAINED_BERT_CACHEÚPYTORCH_TRANSFORMERS_CACHEÚTRANSFORMERS_CACHEa—In Transformers v4.0.0, the default path to cache downloaded models changed from '~/.cache/torch/transformers' to '~/.cache/huggingface/transformers'. Since you don't seem to have overridden and '~/.cache/torch/transformers' is a directory that exists, we're moving it to '~/.cache/huggingface/transformers' to avoid redownloading models you have already in the cache. You should only see this message once.ÚHUGGINGFACE_HUB_CACHEÚHF_MODULES_CACHEÚmodulesZtransformers_modulesÚDISABLE_TELEMETRYz3https://s3.amazonaws.com/models.huggingface.co/bertzhttps://cdn.huggingface.coZHUGGINGFACE_CO_STAGINGÚNOzhttps://hub-ci.huggingface.cozhttps://huggingface.coÚHUGGINGFACE_CO_RESOLVE_ENDPOINTz‘Using the environment variable `HUGGINGFACE_CO_RESOLVE_ENDPOINT` is deprecated and will be removed in Transformers v5. Use `HF_ENDPOINT` instead.Z HF_ENDPOINTz)/{model_id}/resolve/{revision}/{filename}z/api/telemetry/examplescCst|ƒ}|jdkS)N)ÚhttpÚhttps)r Úscheme)Zurl_or_filenameÚparsedr*r*r+Ú is_remote_url„sr<)Ú cache_dirÚreturnc CsÂ|dkrt}nt|tƒr t|ƒ}tj |¡s0gSg}t |¡D]~}| d¡r>tj  ||¡}t |ddP}t   |¡}|d}|d}| d¡r²tj  | d¡¡d}| |||f¡W5QRXq>|S) a  Returns a list of tuples representing model binaries that are cached locally. Each tuple has shape `(model_url, etag, size_MB)`. Filenames in `cache_dir` are use to get the metadata for each model, only urls ending with *.bin* are added. Args: cache_dir (`Union[str, Path]`, *optional*): The cache directory to search for models within. Will default to the transformers cache if unset. Returns: List[Tuple]: List of tuples each with shape `(model_url, etag, size_MB)` Nú.jsonúutf-8©ÚencodingÚurlÚetagz.bing€„.A)r1Ú isinstancerÚstrÚosÚpathÚisdirÚlistdirÚendswithÚjoinÚopenÚjsonÚloadÚgetsizeÚstripÚappend) r=Z cached_modelsÚfileÚ meta_pathÚ meta_fileÚmetadatarCrDZsize_MBr*r*r+Úget_cached_models‰s$      rWc CsÎz2t tjd¡ ¡}|d}|d d¡d}Wntk rNd}d}YnXt t dd¡¡}d|krndnd }d tjkrt d ¡ d¡d nd}t d d¡t d d¡t dd¡t dd¡||||dœ}|S)NZECS_CONTAINER_METADATA_URIZImageú:rZSM_FRAMEWORK_PARAMSz{}Z*sagemaker_distributed_dataparallel_enabledTFZTRAINING_JOB_ARNéZSM_FRAMEWORK_MODULEZ AWS_REGIONZ SM_NUM_GPUSrZ SM_NUM_CPUS)Z sm_frameworkZ sm_regionZ sm_number_gpuZ sm_number_cpuZsm_distributed_trainingZsm_deep_learning_containerZsm_deep_learning_container_tagZ sm_account_id) ÚrequestsÚgetrGÚenvironrNÚsplitÚ ExceptionÚloadsÚgetenv)Z instance_dataZdlc_container_usedZdlc_tagZsagemaker_paramsZruns_distributed_trainingZ account_idZsagemaker_objectr*r*r+Údefine_sagemaker_information¬s( "    ø ra)Ú user_agentr>cCsàdt›dtj ¡d›dt›}tƒr4|dt›7}tƒrH|dt›7}t rT|dSt ƒr||dd  d d „t ƒ  ¡Dƒ¡7}tj d d ¡ ¡tkrš|d 7}t|tƒrÆ|dd  dd „|  ¡Dƒ¡7}nt|tƒrÜ|d|7}|S)zF Formats a user-agent string with basic info about a request. z transformers/z ; python/rz ; session_id/z; torch/z ; tensorflow/z; telemetry/offz; css |]\}}|›d|›VqdS©ú/Nr*©Ú.0ÚkÚvr*r*r+Ú Òsz"http_user_agent..ZTRANSFORMERS_IS_CIÚz ; is_ci/truecss |]\}}|›d|›VqdSrcr*rer*r*r+ri×s)rÚsysÚversionr]Ú SESSION_IDr$r"r#r!r5r%rLraÚitemsrGr\r[Úupperr rEÚdictrF)rbZuar*r*r+Úhttp_user_agentÆs  " "  rq)Ú resolved_fileÚ commit_hashcCsZ|dks|dk r|Stt|ƒ ¡ƒ}t d|¡}|dkrc Cs*|dkr d}|dkrt}| dd¡}|dkr0d}tj ||›d|›¡}tj |¡sXdSdD] }tj tj ||¡¡s\dSq\t tj |d¡¡}||krÂttj |d|¡ƒ} |  ¡}W5QRXtj  tj |d ||¡¡ràt St tj |d ¡¡} || krdStj |d ||¡} tj  | ¡r&| SdS) a  Explores the cache to return the latest cached file for a given revision if found. This function will not raise any exception if the file in not cached. Args: cache_dir (`str` or `os.PathLike`): The folder where the cached files lie. repo_id (`str`): The ID of the repo on huggingface.co. filename (`str`): The filename to look for inside `repo_id`. revision (`str`, *optional*): The specific model version to use. Will default to `"main"` if it's not provided and no `commit_hash` is provided either. repo_type (`str`, *optional*): The type of the repo. Returns: `Optional[str]` or `_CACHED_NO_EXIST`: Will return `None` if the file was not cached. Otherwise: - The exact path to the cached file if it's found in the cache - A special value `_CACHED_NO_EXIST` if the file does not exist at the given commit hash and this fact was cached. NÚmainrdú--Úmodelzs--)ÚrefsÚ snapshotsrz .no_existr‚) r1ÚreplacerGrHrLrIrJrMÚreadÚisfileÚ_CACHED_NO_EXIST) rzr{r=r|r}Z object_idZ repo_cacheÚ subfolderZ cached_refsÚfZ cached_shasÚ cached_filer*r*r+Útry_to_load_from_cacheës0    rŠrj)Úpath_or_repo_idr{r=Úforce_downloadÚresume_downloadÚproxiesÚtokenr|Úlocal_files_onlyr‡r}rbÚ%_raise_exceptions_for_missing_entriesÚ'_raise_exceptions_for_connection_errorsÚ _commit_hashcKsŽ| dd¡}|dk r4t dt¡|dk r0tdƒ‚|}tƒrL|sLt d¡d}| dkrXd} t|ƒ}t j   | |¡}t j   |¡rÌt j   t j   || ¡|¡}t j   |¡sÈ| rÄt|›d|›d |›d |›d ƒ‚ndS|S|dkrØt}t|tƒrêt|ƒ}|dk rD|sDt||||| d }|dk rD|tk r$|S| s.dStd |›d|›dƒ‚t| ƒ} z8t||t| ƒdkrfdn| | ||| |||||d }Wntk r¼}ztd|›dƒ|‚W5d}~XYnÎtk rð}zt|›dƒ|‚W5d}~XYnštk r*}zt|›d|›dƒ|‚W5d}~XYn`tk r°}zht||||d}|dk rl|tkrl|WY¢:S| rx| s‚WY¢&dStdt›d|›d|›dƒ|‚W5d}~XYnÚtk r}zD| sÒWY¢8dS|dkràd}t|›d|›d |›d |›d ƒ|‚W5d}~XYnxtk rˆ}zXt||||d}|dk rT|tkrT|WY¢*S| sdWY¢dStd|›d|›ƒ‚W5d}~XYnX|S)aš Tries to locate a file in a local folder and repo, downloads and cache it if necessary. Args: path_or_repo_id (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a model repo on huggingface.co. - a path to a *directory* potentially containing the file. filename (`str`): The name of the file to locate in `path_or_repo`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. resume_download (`bool`, *optional*, defaults to `False`): Whether or not to delete incompletely received file. Attempts to resume the download if such a file exists. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. repo_type (`str`, *optional*): Specify the repo type (useful when downloading from a space for instance). Passing `token=True` is required when you want to use a private model. Returns: `Optional[str]`: Returns the resolved file (to the cache folder if downloaded from a repo). Examples: ```python # Download a model weight from the Hub and cache it. model_weights_file = cached_file("bert-base-uncased", "pytorch_model.bin") ```Úuse_auth_tokenNúVThe `use_auth_token` argument is deprecated and will be removed in v5 of Transformers.úV`token` and `use_auth_token` are both specified. Please set only the argument `token`.z+Offline mode: forcing local_files_only=TrueTrjú& does not appear to have a file named z#. Checkout 'https://huggingface.co/rdz' for available files.)r=r|r}zCould not locate z inside Ú.r) r‡r}r|r=rbrŒrŽrrrz]You are trying to access a gated repo. Make sure to request access at https://huggingface.co/ú† and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=`.a is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=`ú• is not a valid git identifier (branch name, tag name or commit id) that exists for this model name. Check the model page at 'https://huggingface.co/ú' for available revisions.©r=r|úWe couldn't connect to 'zL' to load this file, couldn't find it in the cached files and it looks like z8 is not the path to a directory containing a file named zš. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.r~z:There was a specific connection error when trying to load z: )ÚpopÚwarningsÚwarnÚ FutureWarningÚ ValueErrorr,ÚloggerÚinforFrGrHrLrIr…ÚEnvironmentErrorr1rErrŠr†rqrÚlenrrrrr7rr)r‹r{r=rŒrrŽrr|rr‡r}rbr‘r’r“Údeprecated_kwargsr”Z full_filenamerrÚeÚerrr*r*r+r‰.sÔF ÿ    ÿ ÿ  ô  ÿüÿûÿü   ÿû  ÿý  &r‰) Ú path_or_repor{r=rŒrrŽrr|rr‡c KsT|  dd¡} | dk r4t dt¡|dk r0tdƒ‚| }t|||||||||| ddd S)a Tries to locate a file in a local folder and repo, downloads and cache it if necessary. Args: path_or_repo (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a model repo on huggingface.co. - a path to a *directory* potentially containing the file. filename (`str`): The name of the file to locate in `path_or_repo`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. resume_download (`bool`, *optional*, defaults to `False`): Whether or not to delete incompletely received file. Attempts to resume the download if such a file exists. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. Passing `token=True` is required when you want to use a private model. Returns: `Optional[str]`: Returns the resolved file (to the cache folder if downloaded from a repo) or `None` if the file does not exist. Examples: ```python # Download a tokenizer configuration from huggingface.co and cache. tokenizer_config = get_file_from_repo("bert-base-uncased", "tokenizer_config.json") # This model does not have a tokenizer config so the result will be None. tokenizer_config = get_file_from_repo("xlm-roberta-base", "tokenizer_config.json") ```r”Nr•r–F) r‹r{r=rŒrrŽrr|rr‡r‘r’)ržrŸr r¡r¢r‰) rªr{r=rŒrrŽrr|rr‡r§r”r*r*r+Úget_file_from_repoñs.B ÿôr«c CsFt d|›d¡t ¡d}t|dƒ}t|||dW5QRX|S)aw Downloads a given url in a temporary file. This function is not safe to use in multiple processes. Its only use is for deprecated behavior allowing to download config/models with a single url instead of using the Hub. Args: url (`str`): The url of the file to download. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. Returns: `str`: The location of the temporary file where the url was downloaded. z5Using `from_pretrained` with the url of a file (here aY) is deprecated and won't be possible anymore in v5 of Transformers. You should host your file on the Hub (hf.co) instead and use the repository ID. Note that this is not compatible with the caching system (your file will be downloaded at each execution) or multiple processes (each process will download the file in a different temporary file).rÚwb)rŽ)rŸr ÚtempfileÚmkstemprMr)rCrŽZtmp_filerˆr*r*r+Ú download_urlLs ÿ  r¯)rªr{r|rŽrc Ksn| dd¡}|dk r4t dt¡|dk r0tdƒ‚|}tj |¡rVtj tj  ||¡¡St |||d}t |t ƒd}t j||d|dd } zt| ƒWd Stk rÖ} z$t | ¡t|›d |›d ƒ| ‚W5d} ~ XYn”tk r} zt | ¡t|›d ƒ‚W5d} ~ XYnZtk rP} z"t | ¡t|›d|›dƒ‚W5d} ~ XYnt jk rhYdSXdS)aV Checks if a repo contains a given file without downloading it. Works for remote repos and local folders. This function will raise an error if the repository `path_or_repo` is not valid or if `revision` does not exist for this repo, but will return False for regular connection errors. r”Nr•r–)r{r|)rrbFé )ÚheadersÚallow_redirectsrŽÚtimeoutTzN is a gated repository. Make sure to request access at https://huggingface.co/r™zE is not a local folder or a valid repository name on 'https://hf.co'.ršr›)ržrŸr r¡r¢rGrHrIr…rLrrrqrZÚheadrrr£Úerrorr¥rrr) rªr{r|rŽrr§r”rCr±Úrr¨r*r*r+Úhas_filefsD ÿ  ÿü  ÿr·c @sÞeZdZdZdeeeeeeefeeeeedœdd„Zeee j fdœdd„Z deee j fee ee feeeeeefeed œd d „Zdeeeeeeeeeeefeeeefeeeedœ dd„ZdS)ÚPushToHubMixinzW A Mixin containing the functionality to push a model or tokenizer to the hub. N)rzÚprivaterÚrepo_urlÚ organizationr>cCsŠ|dk r4t d¡|dk r"tdƒ‚| t›dd¡}|dk rtt d¡| |¡std|krf| d¡d}|›d|›}t|||dd }|jS) z‘ Create the repo if needed, cleans up repo_id with deprecated kwargs `repo_url` and `organization`, retrieves the token. NzgThe `repo_url` argument is deprecated and will be removed in v5 of Transformers. Use `repo_id` instead.zT`repo_id` and `repo_url` are both specified. Please set only the argument `repo_id`.rdrjzºThe `organization` argument is deprecated and will be removed in v5 of Transformers. Set your organization directly in the `repo_id` passed instead (`repo_id={organization}/{model_id}`).éÿÿÿÿT)rzrr¹Úexist_ok) rŸr r¢rƒr7Ú startswithr]rrz)Úselfrzr¹rrºr»rCr*r*r+Ú _create_repo¥s& ÿÿÿ zPushToHubMixin._create_repo©Ú working_dircs‡fdd„t ˆ¡DƒS)zS Returns the list of files with their last modification timestamp. c s$i|]}|tj tj ˆ|¡¡“qSr*©rGrHÚgetmtimerL©rfrˆrÁr*r+Ú Ìsz8PushToHubMixin._get_files_timestamps..)rGrJ)r¿rÂr*rÁr+Ú_get_files_timestampsÈsz$PushToHubMixin._get_files_timestampsF)rÂrzÚfiles_timestampsÚcommit_messagerÚ create_prr|c sl|dkrpd|jjkrd}nVd|jjkr,d}nDd|jjkr>d}n2d|jjkrPd }n d |jjkrbd }nd |jj›}‡‡fd d„t ˆ¡Dƒ}‡fdd„|Dƒ}g} |D]|} tj tj ˆ| ¡¡rt tj ˆ| ¡¡D],} |  ttj ˆ| | ¡tj | | ¡d¡qÔq¤|  ttj ˆ| ¡| d¡q¤|dk rèsþz9PushToHubMixin._upload_modified_files..cs<g|]4}tj tj ˆ|¡¡s4tj tj ˆ|¡¡r|‘qSr*)rGrHr…rLrIrÅrÁr*r+rÍïsþ)Zpath_or_fileobjZ path_in_repoT)rzÚbranchrr½z!Uploading the following files to ú: ú,)rzÚ operationsrÉrrÊr|) Ú __class__Ú__name__rGrJrHrIrLrRr r r£r¤r ) r¿rÂrzrÈrÉrrÊr|Zmodified_filesrÑrSrˆr*rÌr+Ú_upload_modified_filesÎsT       þ þ ÿÿÿ úz%PushToHubMixin._upload_modified_filesÚ10GB) rzÚ use_temp_dirrÉr¹rÚmax_shard_sizerÊÚsafe_serializationr|r>c  KsB|  dd¡} | dk r4t dt¡|dk r0tdƒ‚| }|  dd¡} | dk rœt dt¡|dk rdtdƒ‚tj | ¡rˆ| tjj ¡d}|} qª| }| d ¡d} n| d ¡d} |  d d¡}|  d d¡}|j |||||d }|dkrìtj | ¡ }t | |d @}|  |¡}|j |||d|j||||||| dW5QR£SQRXdS)u? Upload the {object_files} to the 🤗 Model Hub. Parameters: repo_id (`str`): The name of the repository you want to push your {object} to. It should contain your organization name when pushing to a given organization. use_temp_dir (`bool`, *optional*): Whether or not to use a temporary directory to store the files saved before they are pushed to the Hub. Will default to `True` if there is no directory named like `repo_id`, `False` otherwise. commit_message (`str`, *optional*): Message to commit while pushing. Will default to `"Upload {object}"`. private (`bool`, *optional*): Whether or not the repository created should be private. token (`bool` or `str`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). Will default to `True` if `repo_url` is not specified. max_shard_size (`int` or `str`, *optional*, defaults to `"10GB"`): Only applicable for models. The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size lower than this size. If expressed as a string, needs to be digits followed by a unit (like `"5MB"`). create_pr (`bool`, *optional*, defaults to `False`): Whether or not to create a PR with the uploaded files or directly commit. safe_serialization (`bool`, *optional*, defaults to `False`): Whether or not to convert the model weights in safetensors format for safer serialization. revision (`str`, *optional*): Branch to push the uploaded files to. Examples: ```python from transformers import {object_class} {object} = {object_class}.from_pretrained("bert-base-cased") # Push the {object} to your namespace with the name "my-finetuned-bert". {object}.push_to_hub("my-finetuned-bert") # Push the {object} to an organization with the name "my-finetuned-bert". {object}.push_to_hub("huggingface/my-finetuned-bert") ``` r”Nr•r–Úrepo_path_or_namezpThe `repo_path_or_name` argument is deprecated and will be removed in v5 of Transformers. Use `repo_id` instead.z]`repo_id` and `repo_path_or_name` are both specified. Please set only the argument `repo_id`.r¼rdrºr»)r¹rrºr»)rÂrÖ)r×rØ)rÉrrÊr|)ržrŸr r¡r¢rGrHrIr]ÚseprÀrrÇZsave_pretrainedrÔ)r¿rzrÖrÉr¹rr×rÊrØr|r§r”rÙrÂrºr»Zwork_dirrÈr*r*r+Ú push_to_hubsd8 ÿÿ ýÿ   ÿ ùzPushToHubMixin.push_to_hub)NNNN)NNFN)NNNNrÕFFN)rÓÚ __module__Ú __qualname__Ú__doc__rFrÚboolrrÀrGÚPathLikerÇrÚfloatrÔÚintrÛr*r*r*r+r¸ s^úù # ø  ø Göôr¸Zpytorch)Ú frameworkc Gsötƒr dS||dœ}|D]œ}dd„|j ¡Dƒ}d|krX|d}tj |¡sX|d|d<d|krn|d|d<qd|kr| d d ¡ d d ¡ d d ¡}| d d ¡}|›d|d›|d<qdt|ƒi}ztj t |d} |   ¡Wnt k rðYnXdS)a² Sends telemetry that helps tracking the examples use. Args: example_name (`str`): The name of the example. *example_args (dataclasses or `argparse.ArgumentParser`): The arguments to the script. This function will only try to extract the model and dataset name from those. Nothing else is tracked. framework (`str`, *optional*, defaults to `"pytorch"`): The framework for the example. N)ZexamplerãcSs(i|] \}}| d¡s|dk r||“qS)Ú_N)r¾rer*r*r+rÆ™s  z*send_example_telemetry..Zmodel_name_or_pathÚ model_nameZ dataset_nameZ task_nameZtf_rjZflax_Zrun_Z _no_trainerú-z user-agent)r±) r,Ú__dict__rnrGrHrIrƒrqrZr´Ú!HUGGINGFACE_CO_EXAMPLES_TELEMETRYÚraise_for_statusr^) Z example_namerãZ example_argsÚdataÚargsZ args_as_dictråÚ script_namer±r¶r*r*r+Úsend_example_telemetryŠs*       rí)ÚsizecCs,t|tƒr|S| ¡ d¡r0t|dd…ƒdS| ¡ d¡rRt|dd…ƒdS| ¡ d¡rtt|dd…ƒdS| ¡ d ¡r¬t|dd …ƒd }| d ¡r¨|d S|S| ¡ d¡rät|dd …ƒd}| d ¡rà|d S|S| ¡ d¡r t|dd …ƒd}| d ¡r|d S|Stdƒ‚dS)a' Converts a size expressed as a string with digits an unit (like `"5MB"`) to an integer (in bytes). Args: size (`int` or `str`): The size to convert. Will be directly returned if an `int`. Example: ```py >>> convert_file_size_to_int("1MiB") 1048576 ``` ZGIBNéýÿÿÿi@ZMIBiZKIBiÚGBéþÿÿÿiÊš;ÚbéÚMBi@BZKBièzR`size` is not in a valid format. Use an integer followed by the unit, e.g., '5GB'.)rErârorKr¢)rîZint_sizer*r*r+Úconvert_file_size_to_int°s$ rõc  s´ddl} |  dd¡}|dk r.r¼rœzDownloading shards)ÚdescÚdisable) r=rŒrŽrrrrbr|r‡r“r—z5 which is required according to the checkpoint index.rz ' to load z?. You should try again after checking your internet connection.)rNržrŸr r¡r¢rGrHr…rMr_r„ÚsortedÚsetÚvaluesÚlistÚkeysÚcopyrIrŠr&r‰rr¥rr7rR)r÷Zindex_filenamer=rŒrŽrrrrbr|r‡r“r§rNr”rˆÚindexZshard_filenamesZsharded_metadataZcached_filenamesZ last_shardZshow_progress_barZshard_filenameZcached_filenamer*rör+Úget_checkpoint_shard_filesÑsn ÿ   ÿô ÿÿ rc Cs¬|dkrt}nt|ƒ}tj |¡s&gSg}t |¡D]r}tj ||›d¡}tj |¡sZq4t|dd:}t   |¡}|d}|d  dd¡}|  |||d œ¡W5QRXq4|S) zH Returns a list for all files cached with appropriate metadata. Nr?r@rArCrDú"rj)rSrCrD) r1rFrGrHrIrJrLr…rMrNrOrƒrR)r=Ú cached_filesrSrTrUrVrCrDr*r*r+Úget_all_cached_files3s    rcCsHt d|¡}|dkrdS| ¡\}}}d dg| d¡¡}|||dœS)z? Extract repo_name, revision and filename from an url. z3^https://huggingface\.co/(.*)/resolve/([^/]*)/(.*)$NrÚmodelsrd)Úrepor|r{)rurvrwrLr])rCrvrr|r{Z cache_repor*r*r+Úextract_info_from_urlMs  rcCs6||›d|›dfD]}tj |¡rt |¡qdS)z> Remove, if they exist, file, file.json and file.lock r?z.lockN)rGrHr…Úremove)rSrˆr*r*r+Úclean_files_forYs r c Csðtj|ddtjtj |d¡dd||kr^tj |d|¡}t|dƒ}| |¡W5QRXtjtj |d¡ddtj |d|¡}t ||¡tjtj |d¡ddtjtj |d|¡ddtj |d||¡} tj   || ¡t |ƒdS)zQ Move file to repo following the new huggingface hub cache organization. T©r½rÚwZblobsr‚N) rGÚmakedirsrHrLrMÚwriteÚshutilÚmoveÚhuggingface_hubZ file_downloadZ_create_relative_symlinkr ) rSrr{r|rDrsZref_pathrˆZ blob_pathZ pointer_pathr*r*r+Úmove_to_new_cachebs  rc CsV|dkr t}|dkr@ttƒjd}tj t|ƒ¡rcCs|dkr gn||_dSr(©r)r¿rr*r*r+Ú__init__°szPushInProgress.__init__cCstdd„|jDƒƒS)Ncss|]}| ¡VqdSr()Údone©rfZjobr*r*r+ri´sz)PushInProgress.is_done..)Úallr©r¿r*r*r+Úis_done³szPushInProgress.is_donecCst |j¡dSr()rÚwaitrrr*r*r+Úwait_until_done¶szPushInProgress.wait_until_done)r>cCsdd„|jDƒ|_dS)NcSs g|]}| ¡s| ¡s|‘qSr*)Úcancelrrr*r*r+rͺsýz)PushInProgress.cancel..rrr*r*r+r ¹sþzPushInProgress.cancel)N) rÓrÜrÝrÞrrÚFuturerrrr r*r*r*r+r«s rz version.txtauYou are offline and the cache for model files in Transformers v4.22.0 has been updated while your local cache seems to be the one of a previous version. It is very likely that all your calls to any `from_pretrained()` method will fail. Remove the offline mode and enable internet connection to have your cache be updated automatically, then you can go back to offline mode.zçThe cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.Ú z5There was a problem when trying to move your cache: rÏzš Please file an issue at https://github.com/huggingface/transformers/issues/new/choose and copy paste this whole message and we will do our best to help.r r Ú1z?There was a problem when trying to write in your cache folder (zV). You should set the environment variable TRANSFORMERS_CACHE to a writable directory.)N)N)NNN) NFFNNNFrjNNTTN)NFFNNNFrj)N)NNN) NFNFFNNNrjN)N)NNN)ŒrÞrNrGrurrkr­Ú tracebackrŸÚ concurrentrÚpathlibrÚtypingrrrrrÚ urllib.parser Úuuidr rrZr r r rrrrZhuggingface_hub.file_downloadrrZhuggingface_hub.utilsrrrrrrrZrequests.exceptionsrrjrrZgenericrZ import_utilsr r!r"r#r$r%r&Z get_loggerrÓr£r\r[ror)r,r`rHrLZtorch_cache_homeZold_default_cache_pathÚ expanduserZ hf_cache_homeZdefault_cache_pathrIÚwarningrr/r0r2r1r3Z TRANSFORMERS_DYNAMIC_MODULE_NAMEÚhexrmr5ZS3_BUCKET_PREFIXZCLOUDFRONT_DISTRIB_PREFIXZ _staging_modeZ_default_endpointr7r r¡ZHUGGINGFACE_CO_PREFIXrèÚobjectr†r<rFrWrarqryrŠràrßr‰r«r¯r·r¸rírârõrrrr rrrZcache_version_filer…Z cache_versionrMrˆr„r¢r¦rJZcache_is_not_emptyr^r¨Ú format_tbÚ __traceback__ÚtracerµrÒr r r*r*r*r+Ús–    $ $      ÿ ÿ þýüûÿ     ý#û ú Fñ ñ Gö ö [ û û :k&$ô b    /  ÿÿ   ÿ   ÿ