U 9%e92@sdZddlZddlZddlmZddlmZmZmZm Z m Z erLddl m Z dede e e e eddd d ZeGd d d ZeGd ddZGdddeZGdddeZGdddeZGdddeZdS)zz Helpful utility functions and classes in relation to exploring API endpoints with the aim for a user-friendly interface. N) dataclass) TYPE_CHECKINGIterableListOptionalUnion) ModelInfor )modelsminimum_thresholdmaximum_thresholdreturnccs|dkr|dkrtd|dkr$d}|dkr2tj}|D]}t|dd}|dks6t|tsZq6|dd}t|trx|d}|s~q6tdt |}|dkrq6t | d}||kr6||kr6|Vq6dS) aFilters a list of models for those that include an emission tag and limit them to between two thresholds Args: models (Iterable of `ModelInfo`): A list of models to filter. minimum_threshold (`float`, *optional*): A minimum carbon threshold to filter by, such as 1. maximum_threshold (`float`, *optional*): A maximum carbon threshold to filter by, such as 10. NzFBoth `minimum_threshold` and `maximum_threshold` cannot both be `None`ZcardDataZco2_eq_emissionsZ emissionsz \d+\.\d+|\d+r) ValueErrormathinfgetattr isinstancedictgetresearchstrfloatgroup)r r r modelZ card_dataZemissionZmatchedZemission_valuere/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/huggingface_hub/utils/endpoint_helpers.py_filter_emissionss*   rc@seZdZUdZdZeeed<dZee ee efed<dZ eeed<dZ ee ee efed<dZ ee ee efed<dZee ee efed<dZee ee efed <dZee ee efed <dZee ee efed <dS) DatasetFiltera A class that converts human-readable dataset search parameters into ones compatible with the REST API. For all parameters capitalization does not matter. Args: author (`str`, *optional*): A string or list of strings that can be used to identify datasets on the Hub by the original uploader (author or organization), such as `facebook` or `huggingface`. benchmark (`str` or `List`, *optional*): A string or list of strings that can be used to identify datasets on the Hub by their official benchmark. dataset_name (`str`, *optional*): A string or list of strings that can be used to identify datasets on the Hub by its name, such as `SQAC` or `wikineural` language_creators (`str` or `List`, *optional*): A string or list of strings that can be used to identify datasets on the Hub with how the data was curated, such as `crowdsourced` or `machine_generated`. language (`str` or `List`, *optional*): A string or list of strings representing a two-character language to filter datasets by on the Hub. multilinguality (`str` or `List`, *optional*): A string or list of strings representing a filter for datasets that contain multiple languages. size_categories (`str` or `List`, *optional*): A string or list of strings that can be used to identify datasets on the Hub by the size of the dataset such as `100K>> from huggingface_hub import DatasetFilter >>> # Using author >>> new_filter = DatasetFilter(author="facebook") >>> # Using benchmark >>> new_filter = DatasetFilter(benchmark="raft") >>> # Using dataset_name >>> new_filter = DatasetFilter(dataset_name="wikineural") >>> # Using language_creator >>> new_filter = DatasetFilter(language_creator="crowdsourced") >>> # Using language >>> new_filter = DatasetFilter(language="en") >>> # Using multilinguality >>> new_filter = DatasetFilter(multilinguality="multilingual") >>> # Using size_categories >>> new_filter = DatasetFilter(size_categories="100K>> # Using task_categories >>> new_filter = DatasetFilter(task_categories="audio_classification") >>> # Using task_ids >>> new_filter = DatasetFilter(task_ids="paraphrase") ``` Nauthor benchmark dataset_namelanguage_creatorslanguagemultilingualitysize_categoriestask_categoriestask_ids)__name__ __module__ __qualname____doc__r rr__annotations__r!rrr"r#r$r%r&r'r(rrrrrFs Irc@seZdZUdZdZeeed<dZee ee efed<dZ ee ee efed<dZ eeed<dZ ee ee efed<dZee ee efed<dZee ee efed <dS) ModelFilteraB A class that converts human-readable model search parameters into ones compatible with the REST API. For all parameters capitalization does not matter. Args: author (`str`, *optional*): A string that can be used to identify models on the Hub by the original uploader (author or organization), such as `facebook` or `huggingface`. library (`str` or `List`, *optional*): A string or list of strings of foundational libraries models were originally trained from, such as pytorch, tensorflow, or allennlp. language (`str` or `List`, *optional*): A string or list of strings of languages, both by name and country code, such as "en" or "English" model_name (`str`, *optional*): A string that contain complete or partial names for models on the Hub, such as "bert" or "bert-base-cased" task (`str` or `List`, *optional*): A string or list of strings of tasks models were designed for, such as: "fill-mask" or "automatic-speech-recognition" tags (`str` or `List`, *optional*): A string tag or a list of tags to filter models on the Hub by, such as `text-generation` or `spacy`. trained_dataset (`str` or `List`, *optional*): A string tag or a list of string tags of the trained dataset for a model on the Hub. ```python >>> from huggingface_hub import ModelFilter >>> # For the author_or_organization >>> new_filter = ModelFilter(author_or_organization="facebook") >>> # For the library >>> new_filter = ModelFilter(library="pytorch") >>> # For the language >>> new_filter = ModelFilter(language="french") >>> # For the model_name >>> new_filter = ModelFilter(model_name="bert") >>> # For the task >>> new_filter = ModelFilter(task="text-classification") >>> # Retrieving tags using the `HfApi.get_model_tags` method >>> from huggingface_hub import HfApi >>> api = HfApi() # To list model tags >>> api.get_model_tags() # To list dataset tags >>> api.get_dataset_tags() >>> new_filter = ModelFilter(tags="benchmark:raft") >>> # Related to the dataset >>> new_filter = ModelFilter(trained_dataset="common_voice") ``` Nr libraryr$ model_nametasktrained_datasettags)r)r*r+r,r rrr-r/rrr$r0r1r2r3rrrrr.s Ar.csDeZdZdZddZfddZddZfdd Zd d ZZ S) AttributeDictionarya- `dict` subclass that also provides access to keys as attributes If a key starts with a number, it will exist in the dictionary but not as an attribute Example: ```python >>> d = AttributeDictionary() >>> d["test"] = "a" >>> print(d.test) # prints "a" ``` cCs||kr||St|dSNAttributeErrorselfkrrr __getattr__szAttributeDictionary.__getattr__cs$|jtjf|ddk||dS)Nr_) __setitem__super __setattr__)r9r:v __class__rrr?szAttributeDictionary.__setattr__cCs||kr||=nt|dSr5r6r8rrr __delattr__szAttributeDictionary.__delattr__cs(t|}dd|D}t|S)NcSs g|]}|ddr|qS)r<)replaceisalpha).0keyrrr sz/AttributeDictionary.__dir__..)sortedkeysr>__dir__)r9rKrArrrLs zAttributeDictionary.__dir__cCsHd}t|D]2}|d|7}|dds:|d7}|d7}q|S)NzAvailable Attributes or Keys: z * r<rDz (Key only) )rJrKrErF)r9repr_strrHrrr__repr__ s zAttributeDictionary.__repr__) r)r*r+r,r;r?rCrLrO __classcell__rrrArr4s   r4c@s4eZdZdZd eeedddZedddZ dS) GeneralTagsa! A namespace object holding all tags, filtered by `keys` If a tag starts with a number, it will only exist in the dictionary Example: ```python >>> a.b["1a"] # will work >>> a["b"]["1a"] # will work >>> # a.b.1a # will not work ``` Args: tag_dictionary (`dict`): A dictionary of tags returned from the /api/***-tags-by-type api endpoint keys (`list`): A list of keys to unpack the `tag_dictionary` with, such as `["library","language"]` N)tag_dictionaryrKcCs4||_|dkrt|j}|D]}||q dSr5)_tag_dictionarylistrK_unpack_and_assign_dictionary)r9rRrKrHrrr__init__,s zGeneralTags.__init__)rHcCs^t}t||||j|gD]0}|ddddddd}|d||<q |||<dS) zYAssign nested attributes to `self.key` containing information as an `AttributeDictionary`label rD-r<.idN)r4setattrrSrrE)r9rHrefitemrWrrrrU3s   z)GeneralTags._unpack_and_assign_dictionary)N) r)r*r+r,rrrTrVrrUrrrrrQsrQcs&eZdZdZedfdd ZZS) ModelTagsa A namespace object holding all available model tags If a tag starts with a number, it will only exist in the dictionary Example: ```python >>> a.dataset["1_5BArabicCorpus"] # will work >>> a["dataset"]["1_5BArabicCorpus"] # will work >>> # o.dataset.1_5BArabicCorpus # will not work ``` Args: model_tag_dictionary (`dict`): A dictionary of valid model tags, returned from the /api/models-tags-by-type api endpoint )model_tag_dictionarycs dddddg}t||dS)Nr/r$licenseZdatasetZ pipeline_tagr>rV)r9r`rKrArrrVPszModelTags.__init__r)r*r+r,rrVrPrrrArr_=sr_cs&eZdZdZedfdd ZZS) DatasetTagsa A namespace object holding all available dataset tags If a tag starts with a number, it will only exist in the dictionary Example ```python >>> a.size_categories["100K>> a["size_categories"]["100K>> # o.size_categories.100K s,   ,UK/&