U 9%es@s ddlZddlmZddlmZddlmZmZmZm Z m Z m Z ddl m Z ddlmZeeZeGdd d ZeGd d d ZGd d d eZGdddeZGdddeZeeeefe eeefdddZddZeeeeeeefdddZdS)N) defaultdict) dataclass)AnyDictListOptionalTupleUnion) yaml_dump) get_loggerc@s eZdZUdZeed<eed<eed<eed<eed<dZeeed<dZ eeed <dZ eeed <dZ eeed <dZ ee eefed <dZeeed <dZeeed<dZee eefed<dZeeed<dZeeed<eedddZdedddZdS) EvalResultu Flattened representation of individual evaluation results found in model-index of Model Cards. For more information on the model-index spec, see https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1. Args: task_type (`str`): The task identifier. Example: "image-classification". dataset_type (`str`): The dataset identifier. Example: "common_voice". Use dataset id from https://hf.co/datasets. dataset_name (`str`): A pretty name for the dataset. Example: "Common Voice (French)". metric_type (`str`): The metric identifier. Example: "wer". Use metric id from https://hf.co/metrics. metric_value (`Any`): The metric value. Example: 0.9 or "20.0 ± 1.2". task_name (`str`, *optional*): A pretty name for the task. Example: "Speech Recognition". dataset_config (`str`, *optional*): The name of the dataset configuration used in `load_dataset()`. Example: fr in `load_dataset("common_voice", "fr")`. See the `datasets` docs for more info: https://hf.co/docs/datasets/package_reference/loading_methods#datasets.load_dataset.name dataset_split (`str`, *optional*): The split used in `load_dataset()`. Example: "test". dataset_revision (`str`, *optional*): The revision (AKA Git Sha) of the dataset used in `load_dataset()`. Example: 5503434ddd753f426f4b38109466949a1217c2bb dataset_args (`Dict[str, Any]`, *optional*): The arguments passed during `Metric.compute()`. Example for `bleu`: `{"max_order": 4}` metric_name (`str`, *optional*): A pretty name for the metric. Example: "Test WER". metric_config (`str`, *optional*): The name of the metric configuration used in `load_metric()`. Example: bleurt-large-512 in `load_metric("bleurt", "bleurt-large-512")`. See the `datasets` docs for more info: https://huggingface.co/docs/datasets/v2.1.0/en/loading#load-configurations metric_args (`Dict[str, Any]`, *optional*): The arguments passed during `Metric.compute()`. Example for `bleu`: max_order: 4 verified (`bool`, *optional*): Indicates whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. Automatically computed by Hugging Face, do not set. verify_token (`str`, *optional*): A JSON Web Token that is used to verify whether the metrics originate from Hugging Face's [evaluation service](https://huggingface.co/spaces/autoevaluate/model-evaluator) or not. task_type dataset_type dataset_name metric_type metric_valueN task_namedataset_config dataset_splitdataset_revision dataset_args metric_name metric_config metric_argsverified verify_tokenreturncCs|j|j|j|j|jfS)z9Returns a tuple that uniquely identifies this evaluation.)rrrrrselfr!\/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/huggingface_hub/repocard_data.pyunique_identifier|s zEvalResult.unique_identifier)otherrcCsD|jD]4\}}|dkrq |dkr t||t||kr dSq dS)zx Return True if `self` and `other` describe exactly the same metric but with a different value. rrFT)__dict__itemsgetattr)r r$key_r!r!r"is_equal_except_values z EvalResult.is_equal_except_value)__name__ __module__ __qualname____doc__str__annotations__rrrrrrrrrrrrboolrpropertytupler#r*r!r!r!r"r s& / r c@seZdZdZdedddZeeefdddZ d d Z dedd d Z ddZ deeedddZ d eeedddZeedddZeed dddZeedddZd S)!CardDataaStructure containing metadata from a RepoCard. [`CardData`] is the parent class of [`ModelCardData`] and [`DatasetCardData`]. Metadata can be exported as a dictionary or YAML. Export can be customized to alter the representation of the data (example: flatten evaluation results). `CardData` behaves as a dictionary (can get, pop, set values) but do not inherit from `dict` to allow this export step. F)ignore_metadata_errorscKs|j|dSN)r%update)r r5kwargsr!r!r"__init__szCardData.__init__rcCst|j}||t|S)zConverts CardData to a dict. Returns: `dict`: CardData represented as a dictionary ready to be dumped to a YAML block for inclusion in a README.md file. )copydeepcopyr%_to_dict _remove_noner Z data_dictr!r!r"to_dicts  zCardData.to_dictcCsdS)zUse this method in child classes to alter the dict representation of the data. Alter the dict in-place. Args: data_dict (`dict`): The raw dict representation of the card data. Nr!r>r!r!r"r<szCardData._to_dictNcCst|d|dS)a Dumps CardData to a YAML block for inclusion in a README.md file. Args: line_break (str, *optional*): The line break to use when dumping to yaml. Returns: `str`: CardData represented as a YAML block. F) sort_keys line_break)r r?strip)r rAr!r!r"to_yamls zCardData.to_yamlcCs|Sr6)rCrr!r!r"__repr__szCardData.__repr__)r(defaultrcCs|j||Sz#Get value for a given metadata key.)r%getr r(rEr!r!r"rGsz CardData.getcCs|j||S)z#Pop value for a given metadata key.)r%poprHr!r!r"rIsz CardData.pop)r(rcCs |j|SrFr%r r(r!r!r" __getitem__szCardData.__getitem__)r(valuercCs||j|<dS)z#Set value for a given metadata key.NrJ)r r(rMr!r!r" __setitem__szCardData.__setitem__cCs ||jkS)z%Check if a given metadata key is set.rJrKr!r!r" __contains__szCardData.__contains__)F)N)N)N)r+r,r-r.r1r9rr/rr?r<rCrDrGrIrLrNrOr!r!r!r"r4s   r4c seZdZdZdddddddddd eeeeefeeeeeeeeeeeeeeeeeee d fddZ ddZ Z S) ModelCardDataa Model Card Metadata that is used by Hugging Face Hub when included at the top of your README.md Args: language (`Union[str, List[str]]`, *optional*): Language of model's training data or metadata. It must be an ISO 639-1, 639-2 or 639-3 code (two/three letters), or a special value like "code", "multilingual". Defaults to `None`. license (`str`, *optional*): License of this model. Example: apache-2.0 or any license from https://huggingface.co/docs/hub/repositories-licenses. Defaults to None. library_name (`str`, *optional*): Name of library used by this model. Example: keras or any library from https://github.com/huggingface/hub-docs/blob/main/js/src/lib/interfaces/Libraries.ts. Defaults to None. tags (`List[str]`, *optional*): List of tags to add to your model that can be used when filtering on the Hugging Face Hub. Defaults to None. datasets (`List[str]`, *optional*): List of datasets that were used to train this model. Should be a dataset ID found on https://hf.co/datasets. Defaults to None. metrics (`List[str]`, *optional*): List of metrics used to evaluate this model. Should be a metric name that can be found at https://hf.co/metrics. Example: 'accuracy'. Defaults to None. eval_results (`Union[List[EvalResult], EvalResult]`, *optional*): List of `huggingface_hub.EvalResult` that define evaluation results of the model. If provided, `model_name` is used to as a name on PapersWithCode's leaderboards. Defaults to `None`. model_name (`str`, *optional*): A name for this model. It is used along with `eval_results` to construct the `model-index` within the card's metadata. The name you supply here is what will be used on PapersWithCode's leaderboards. If None is provided then the repo name is used as a default. Defaults to None. ignore_metadata_errors (`str`): If True, errors while parsing the metadata section will be ignored. Some information might be lost during the process. Use it at your own risk. kwargs (`dict`, *optional*): Additional metadata that will be added to the model card. Defaults to None. Example: ```python >>> from huggingface_hub import ModelCardData >>> card_data = ModelCardData( ... language="en", ... license="mit", ... library_name="timm", ... tags=['image-classification', 'resnet'], ... ) >>> card_data.to_dict() {'language': 'en', 'license': 'mit', 'library_name': 'timm', 'tags': ['image-classification', 'resnet']} ``` NF) languagelicense library_nametagsdatasetsmetrics eval_results model_namer5c s||_||_||_||_||_||_||_||_| dd} | rzt | \}}||_||_WnBt k r} z$| r~t dnt d| dW5d} ~ XYnXtjf| |jrt|jtkr|jg|_|jdkrt ddS)N model-indexzr!r!r"r<?s zModelCardData._to_dict) r+r,r-r.rr r/rr r1r9r< __classcell__r!r!rcr"rPs,6    /rPcs eZdZdZdddddddddddddddeeeeefeeeeefeeeeefeeeeefeeeeefeeeeefeeeeeeeefeeeeefeeeeeeeeeeefe dfddZ ddZ Z S) DatasetCardDataa Dataset Card Metadata that is used by Hugging Face Hub when included at the top of your README.md Args: language (`List[str]`, *optional*): Language of dataset's data or metadata. It must be an ISO 639-1, 639-2 or 639-3 code (two/three letters), or a special value like "code", "multilingual". license (`Union[str, List[str]]`, *optional*): License(s) of this dataset. Example: apache-2.0 or any license from https://huggingface.co/docs/hub/repositories-licenses. annotations_creators (`Union[str, List[str]]`, *optional*): How the annotations for the dataset were created. Options are: 'found', 'crowdsourced', 'expert-generated', 'machine-generated', 'no-annotation', 'other'. language_creators (`Union[str, List[str]]`, *optional*): How the text-based data in the dataset was created. Options are: 'found', 'crowdsourced', 'expert-generated', 'machine-generated', 'other' multilinguality (`Union[str, List[str]]`, *optional*): Whether the dataset is multilingual. Options are: 'monolingual', 'multilingual', 'translation', 'other'. size_categories (`Union[str, List[str]]`, *optional*): The number of examples in the dataset. Options are: 'n<1K', '1K1T', and 'other'. source_datasets (`List[str]]`, *optional*): Indicates whether the dataset is an original dataset or extended from another existing dataset. Options are: 'original' and 'extended'. task_categories (`Union[str, List[str]]`, *optional*): What categories of task does the dataset support? task_ids (`Union[str, List[str]]`, *optional*): What specific tasks does the dataset support? paperswithcode_id (`str`, *optional*): ID of the dataset on PapersWithCode. pretty_name (`str`, *optional*): A more human-readable name for the dataset. (ex. "Cats vs. Dogs") train_eval_index (`Dict`, *optional*): A dictionary that describes the necessary spec for doing evaluation on the Hub. If not provided, it will be gathered from the 'train-eval-index' key of the kwargs. config_names (`Union[str, List[str]]`, *optional*): A list of the available dataset configs for the dataset. NF)rQrRannotations_creatorslanguage_creatorsmultilingualitysize_categoriessource_datasetstask_categoriestask_idspaperswithcode_id pretty_nametrain_eval_index config_namesr5c sl||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ | pV| dd|_ tjf|dS)Ntrain-eval-index)rhrirQrRrjrkrlrmrnrorprrrIrqr_r9)r rQrRrhrirjrkrlrmrnrorprqrrr5r8rcr!r"r9nszDatasetCardData.__init__cCs|d|d<dS)Nrqrs)rIr>r!r!r"r<szDatasetCardData._to_dict) r+r,r-r.rr r/rrr1r9r<rfr!r!rcr"rgFs@* $rgcseZdZdZddddddddddddd eeeeeeeeeeeeeeeeeeeeeeeeeed fddZ Z S) SpaceCardDataa Space Card Metadata that is used by Hugging Face Hub when included at the top of your README.md To get an exhaustive reference of Spaces configuration, please visit https://huggingface.co/docs/hub/spaces-config-reference#spaces-configuration-reference. Args: title (`str`, *optional*) Title of the Space. sdk (`str`, *optional*) SDK of the Space (one of `gradio`, `streamlit`, `docker`, or `static`). sdk_version (`str`, *optional*) Version of the used SDK (if Gradio/Streamlit sdk). python_version (`str`, *optional*) Python version used in the Space (if Gradio/Streamlit sdk). app_file (`str`, *optional*) Path to your main application file (which contains either gradio or streamlit Python code, or static html code). Path is relative to the root of the repository. app_port (`str`, *optional*) Port on which your application is running. Used only if sdk is `docker`. license (`str`, *optional*) License of this model. Example: apache-2.0 or any license from https://huggingface.co/docs/hub/repositories-licenses. duplicated_from (`str`, *optional*) ID of the original Space if this is a duplicated Space. models (List[`str`], *optional*) List of models related to this Space. Should be a dataset ID found on https://hf.co/models. datasets (`List[str]`, *optional*) List of datasets related to this Space. Should be a dataset ID found on https://hf.co/datasets. tags (`List[str]`, *optional*) List of tags to add to your Space that can be used when filtering on the Hub. ignore_metadata_errors (`str`): If True, errors while parsing the metadata section will be ignored. Some information might be lost during the process. Use it at your own risk. kwargs (`dict`, *optional*): Additional metadata that will be added to the space card. Example: ```python >>> from huggingface_hub import SpaceCardData >>> card_data = SpaceCardData( ... title="Dreambooth Training", ... license="mit", ... sdk="gradio", ... duplicated_from="multimodalart/dreambooth-training" ... ) >>> card_data.to_dict() {'title': 'Dreambooth Training', 'sdk': 'gradio', 'license': 'mit', 'duplicated_from': 'multimodalart/dreambooth-training'} ``` NF) titlesdk sdk_versionpython_versionapp_fileapp_portrRduplicated_frommodelsrUrTr5c  sT||_||_||_||_||_||_||_||_| |_| |_ | |_ t j f| dSr6) rurvrwrxryrzrRr{r|rUrTr_r9)r rurvrwrxryrzrRr{r|rUrTr5r8rcr!r"r9szSpaceCardData.__init__) r+r,r-r.rr/intrr1r9rfr!r!rcr"rts64   rt)rarcCsg}|D]}|d}|d}|D]}|dd}|dd}|dd}|dd} |dd} |dd} |dd} |dd } |d D]v}|d}|d }|d}|d }|d}|d }|d }t||| |||| | | | |||||d}||qq"q||fS)aTakes in a model index and returns the model name and a list of `huggingface_hub.EvalResult` objects. A detailed spec of the model index can be found here: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1 Args: model_index (`List[Dict[str, Any]]`): A model index data structure, likely coming from a README.md file on the Hugging Face Hub. Returns: model_name (`str`): The name of the model as found in the model index. This is used as the identifier for the model on leaderboards like PapersWithCode. eval_results (`List[EvalResult]`): A list of `huggingface_hub.EvalResult` objects containing the metrics reported in the provided model_index. Example: ```python >>> from huggingface_hub.repocard_data import model_index_to_eval_results >>> # Define a minimal model index >>> model_index = [ ... { ... "name": "my-cool-model", ... "results": [ ... { ... "task": { ... "type": "image-classification" ... }, ... "dataset": { ... "type": "beans", ... "name": "Beans" ... }, ... "metrics": [ ... { ... "type": "accuracy", ... "value": 0.9 ... } ... ] ... } ... ] ... } ... ] >>> model_name, eval_results = model_index_to_eval_results(model_index) >>> model_name 'my-cool-model' >>> eval_results[0].task_type 'image-classification' >>> eval_results[0].metric_type 'accuracy' ``` nameresultstaskr`datasetconfigsplitrevisionargsrVrMr verifyToken)rrrrrrrrrrrrrrr)rGr append)rarWelemr~rresultrrrrrrrrZmetricrrrrrrr eval_resultr!r!r"rZsP8          rZcCsRt|tttfr&t|dd|DSt|trJt|dd|DS|SdS)zk Recursively remove `None` values from a dict. Borrowed from: https://stackoverflow.com/a/20558778 css|]}|dk rt|VqdSr6r=).0xr!r!r" Psz_remove_none..css2|]*\}}|dk r|dk rt|t|fVqdSr6r)rkvr!r!r"rRsN) isinstancelistr3setr`dictr&)objr!r!r"r=Ks  r=)rXrWrc Cstt}|D]}||j|q g}|D]P}|d}|j|jd|j|j|j |j |j |j ddd|Dd}||q.||dg}t |S)aTakes in given model name and list of `huggingface_hub.EvalResult` and returns a valid model-index that will be compatible with the format expected by the Hugging Face Hub. Args: model_name (`str`): Name of the model (ex. "my-cool-model"). This is used as the identifier for the model on leaderboards like PapersWithCode. eval_results (`List[EvalResult]`): List of `huggingface_hub.EvalResult` objects containing the metrics to be reported in the model-index. Returns: model_index (`List[Dict[str, Any]]`): The eval_results converted to a model-index. Example: ```python >>> from huggingface_hub.repocard_data import eval_results_to_model_index, EvalResult >>> # Define minimal eval_results >>> eval_results = [ ... EvalResult( ... task_type="image-classification", # Required ... dataset_type="beans", # Required ... dataset_name="Beans", # Required ... metric_type="accuracy", # Required ... metric_value=0.9, # Required ... ) ... ] >>> eval_results_to_model_index("my-cool-model", eval_results) [{'name': 'my-cool-model', 'results': [{'task': {'type': 'image-classification'}, 'dataset': {'name': 'Beans', 'type': 'beans'}, 'metrics': [{'type': 'accuracy', 'value': 0.9}]}]}] ``` r)r`r~)r~r`rrrrc Ss.g|]&}|j|j|j|j|j|j|jdqS))r`rMr~rrrr)rrrrrrr)rrr!r!r" s z/eval_results_to_model_index..)rrrV)r~r)rrr#rvaluesrrrrrrrrr=) rXrWZtask_and_ds_types_maprZmodel_index_datarZ sample_resultdatarar!r!r"reWs4%   re)r: collectionsr dataclassesrtypingrrrrrr Zhuggingface_hub.utilsr Z utils.loggingr r+r\r r4rPrgrtr/rZr=rer!r!r!r"s"     EjPQ(d