U -e@sl ddlmZddlmZmZmZmZddlmZddl m Z m Z gZ eGdddZ eGdd d e Ze d d d d d d d ddgdddddddddddddddddddZde_edd d d d d d ddgddddddddddddddde dddd Zd!e_ed"d d d d d d ddgddddddddddddddde dddd#Zd$e_ed%d d d d d d ddgddddddddddddddde dddd#Zd&e_e d'd d d d d d ddgdd(dddd)ddd*dddd+dddddd,Zd-e_ed.d d d d d d ddgdd(dddd)ddd*dddd+dde dddd#Zd/e_ed0d d d d d d ddgdd(dddd)ddd*dddd+dde dddd#Zd1e_ed2d d d d d d ddgdd(dddd)ddd*dddd+dde dddd#Zd3e_e d4d5d d d d d ddgd6d(dddd)ddd*ddd6ddddd6dd,Zd7e_ed8d5d d d d d ddgd6d(dddd)ddd*ddd6ddde dd6dd#Zd9e_ed:d5d d d d d ddgd6d(dddd)ddd*ddd6ddde dd6dd#Zd;e_edd5d d d d d ddgd6d(dddd)ddd*ddd6ddddd6dd,Zd?e_e d@d d d d d d ddgddddddddddddddddddd,ZdAe_e dBd5d d d d d ddgdd(dddd)ddd*ddd6ddddd6dd,ZdCe_e dDd5d d d d d ddgddEddddFdddGddd6ddddd6dd,Z dHe _edId5d d d d d ddgdd(dddd)ddd*ddd6ddde dd6dd#Z!dJe!_edKd5d d d d d ddgddEddddFdddGddd6ddde dd6dd#Z"dLe"_edMd d d d d d ddgddddddddddddddNde #dddOddPZ$dQe$_edRd d d d d d ddgddddddddddddddSde %dddTddPZ&dUe&_edVd d d d d d ddgddddddddddddddWde 'dddOddPZ(dXe(_edYd d d d d d ddgddddddddddddddZde )dddd#Z*d[e*_ed\d d d d d d ddgdddddddddddddd]de +ddd^ddPZ,d_e,_e d`d d d d d d ddgddddddddadbddddddddcdddddeZ-dfe-_e dgd d d d d d ddgddddddddadbddddddddcdddddeZ.dhe._e did5d d d d d ddgdd(dddd)ddadbdd*ddddddcdddd6deZ/dje/_e dkd5d d d d d ddgd6d(dddd)ddd*ddd6dddddd6deZ0dle0_e dmd5d d d d d ddgd6dEddddFdddGddd6dddddd6deZ1dne1_e dod5d d d d d ddgd6dpddddFdddqddd6dddddd6deZ2dre2_eGdsdtdteZ3e3dud5d d d d d ddgd6d(dddd)ddd*ddd6ddSde 4dd6dd#Z5dve5_dS)w) dataclass)AnyDictOptionalTuple)Module)alignerutilsc@speZdZUdZeed<eeefed<eed<e ed<eed<e eddd Z d d Z d d e dddZd S)Wav2Vec2BundleuData class that bundles associated information to use pretrained :py:class:`~torchaudio.models.Wav2Vec2Model`. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - Feature Extraction >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.HUBERT_BASE >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 360M/360M [00:06<00:00, 60.6MB/s] >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Extract acoustic features >>> features, _ = model.extract_features(waveform) _path_params _sample_rate_normalize_waveform _model_typereturncCs|jS)zUSample rate of the audio that the model is trained on. :type: float )rselfrd/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/torchaudio/pipelines/_wav2vec2/impl.py sample_rate1szWav2Vec2Bundle.sample_ratecCst|j|SN)r _get_state_dictr r dl_kwargsrrrr9szWav2Vec2Bundle._get_state_dictNrcCsDt|j|j}||}|||jr8tj|dd}||S)aConstruct the model and load the pretrained weight. The weight file is downloaded from the internet and cached with :func:`torch.hub.load_state_dict_from_url` Args: dl_kwargs (dictionary of keyword arguments): Passed to :func:`torch.hub.load_state_dict_from_url`. Returns: Variation of :py:class:`~torchaudio.models.Wav2Vec2Model`. For the models listed below, an additional layer normalization is performed on the input. For all other models, a :py:class:`~torchaudio.models.Wav2Vec2Model` instance is returned. - WAV2VEC2_LARGE_LV60K - WAV2VEC2_ASR_LARGE_LV60K_10M - WAV2VEC2_ASR_LARGE_LV60K_100H - WAV2VEC2_ASR_LARGE_LV60K_960H - WAV2VEC2_XLSR53 - WAV2VEC2_XLSR_300M - WAV2VEC2_XLSR_1B - WAV2VEC2_XLSR_2B - HUBERT_LARGE - HUBERT_XLARGE - HUBERT_ASR_LARGE - HUBERT_ASR_XLARGE - WAVLM_LARGE T)normalize_waveform) r _get_modelrr rload_state_dictr _extend_modeleval)rrmodel state_dictrrr get_model=s  zWav2Vec2Bundle.get_model)__name__ __module__ __qualname____doc__str__annotations__rrfloatboolpropertyrrrr$rrrrr s r c@s\eZdZUdZeedfed<dZeedfed<ddeeedfdd d Z d d Z d S)Wav2Vec2ASRBundleuData class that bundles associated information to use pretrained :py:class:`~torchaudio.models.Wav2Vec2Model`. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - ASR >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.HUBERT_ASR_LARGE >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 1.18G/1.18G [00:17<00:00, 73.8MB/s] >>> >>> # Check the corresponding labels of the output. >>> labels = bundle.get_labels() >>> print(labels) ('-', '|', 'E', 'T', 'A', 'O', 'N', 'I', 'H', 'S', 'R', 'D', 'L', 'U', 'M', 'W', 'C', 'F', 'G', 'Y', 'P', 'B', 'V', 'K', "'", 'X', 'J', 'Q', 'Z') >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Infer the label probability distribution >>> emissions, _ = model(waveform) >>> >>> # Pass emission to decoder >>> # `ctc_decode` is for illustration purpose only >>> transcripts = ctc_decode(emissions, labels) ._labelsr_remove_aux_axis-blank)r6rcCs |f|jS)auThe output class labels. The first is blank token, and it is customizable. Args: blank (str, optional): Blank token. (default: ``'-'``) Returns: Tuple[str, ...]: For models fine-tuned on ASR, returns the tuple of strings representing the output class labels. Example >>> from torchaudio.pipelines import HUBERT_ASR_LARGE as bundle >>> bundle.get_labels() ('-', '|', 'E', 'T', 'A', 'O', 'N', 'I', 'H', 'S', 'R', 'D', 'L', 'U', 'M', 'W', 'C', 'F', 'G', 'Y', 'P', 'B', 'V', 'K', "'", 'X', 'J', 'Q', 'Z') )r/)rr6rrr get_labelsszWav2Vec2ASRBundle.get_labelscCst|j||jSr)r rr r3rrrrrsz!Wav2Vec2ASRBundle._get_state_dictN) r%r&r'r(rr)r*r3intr7rrrrrr.ds ' r.zwav2vec2_fairseq_base_ls960.pthZ group_norm) )r9r2r1)r9r1r1Fig? i gg?N)extractor_modeextractor_conv_layer_configextractor_conv_biasencoder_embed_dimencoder_projection_dropoutencoder_pos_conv_kernelencoder_pos_conv_groupsencoder_num_layersencoder_num_headsencoder_attention_dropoutencoder_ff_interm_featuresencoder_ff_interm_dropoutencoder_dropoutencoder_layer_norm_firstencoder_layer_drop aux_num_outi>ZWav2Vec2)r r rrraWav2vec 2.0 model ("base" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), not fine-tuned. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ll10m.pth)r r r/rrra9Wav2vec 2.0 model ("base" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 10 minutes of transcribed audio from *Libri-Light* dataset :cite:`librilight` ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ls100.pth)r/rrraWav2vec 2.0 model ("base" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 100 hours of transcribed audio from "train-clean-100" subset. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)wav2vec2_fairseq_base_ls960_asr_ls960.pthaWav2vec 2.0 model ("base" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on the same audio with the corresponding transcripts. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z wav2vec2_fairseq_large_ls960.pthiig?)rrraWav2vec 2.0 model ("large" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), not fine-tuned. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ll10m.ptha:Wav2vec 2.0 model ("large" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 10 minutes of transcribed audio from *Libri-Light* dataset :cite:`librilight` ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ls100.ptha%Wav2vec 2.0 model ("large" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on 100 hours of transcribed audio from the same dataset ("train-clean-100" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_ls960_asr_ls960.pthaWav2vec 2.0 model ("large" architecture with an extra linear module), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), and fine-tuned for ASR on the same audio with the corresponding transcripts. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z wav2vec2_fairseq_large_lv60k.pthZ layer_normTafWav2vec 2.0 model ("large-lv60k" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, not fine-tuned. Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ll10m.pthaWav2vec 2.0 model ("large-lv60k" architecture with an extra linear module), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 10 minutes of transcribed audio from the same dataset ("train-10min" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ls100.pthaWav2vec 2.0 model ("large-lv60k" architecture with an extra linear module), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 100 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` ("train-clean-100" subset). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z*wav2vec2_fairseq_large_lv60k_asr_ls960.ptha+Wav2vec 2.0 model ("large-lv60k" architecture with an extra linear module), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* :cite:`librilight` dataset, and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *wav2vec 2.0* :cite:`baevski2020wav2vec` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z!wav2vec2_fairseq_large_xlsr53.ptha Wav2vec 2.0 model ("base" architecture), pre-trained on 56,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common` and *BABEL* :cite:`Gales2014SpeechRA`), not fine-tuned. Originally published by the authors of *Unsupervised Cross-lingual Representation Learning for Speech Recognition* :cite:`conneau2020unsupervised` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_base_ls960.pthaHuBERT model ("base" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"), not fine-tuned. Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_large_ll60k.pthacHuBERT model ("large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, not fine-tuned. Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zhubert_fairseq_xlarge_ll60k.pthi0iaiHuBERT model ("extra large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, not fine-tuned. Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. z(hubert_fairseq_large_ll60k_asr_ls960.ptha HuBERT model ("large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z)hubert_fairseq_xlarge_ll60k_asr_ls960.pthaHuBERT model ("extra large" architecture), pre-trained on 60,000 hours of unlabeled audio from *Libri-Light* dataset :cite:`librilight`, and fine-tuned for ASR on 960 hours of transcribed audio from *LibriSpeech* dataset :cite:`7178964` (the combination of "train-clean-100", "train-clean-360", and "train-other-500"). Originally published by the authors of *HuBERT* :cite:`hsu2021hubert` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_de.pt )rr1r2#)r/rrr3rawav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 282 hours of transcribed audio from "de" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_en.pt)rr1r2awav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 543 hours of transcribed audio from "en" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_es.ptrSawav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 166 hours of transcribed audio from "es" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_fr.pt+awav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 211 hours of transcribed audio from "fr" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. z%wav2vec2_voxpopuli_base_10k_asr_it.pt%r0awav2vec 2.0 model ("base" architecture), pre-trained on 10k hours of unlabeled audio from *VoxPopuli* dataset :cite:`voxpopuli` ("10k" subset, consisting of 23 languages), and fine-tuned for ASR on 91 hours of transcribed audio from "it" subset. Originally published by the authors of *VoxPopuli* :cite:`voxpopuli` under CC BY-NC 4.0 and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2ASRBundle` for the usage. zwavlm_base.pthi i@)r?r@rArBrCrDrErFrGZencoder_max_distanceZencoder_num_bucketsrHrIrJrKrLrMrNZWavLM)rrraBWavLM Base model ("base" architecture), pre-trained on 960 hours of unlabeled audio from *LibriSpeech* dataset :cite:`7178964`, not fine-tuned. Originally published by the authors of *WavLM* :cite:`chen2022wavlm` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zwavlm_base_plus.pthaWavLM Base+ model ("base" architecture), pre-trained on 60,000 hours of Libri-Light dataset :cite:`librilight`, 10,000 hours of GigaSpeech :cite:`GigaSpeech2021`, and 24,000 hours of *VoxPopuli* :cite:`voxpopuli`, not fine-tuned. Originally published by the authors of *WavLM* :cite:`chen2022wavlm` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zwavlm_large.pthaWavLM Large model ("large" architecture), pre-trained on 60,000 hours of Libri-Light dataset :cite:`librilight`, 10,000 hours of GigaSpeech :cite:`GigaSpeech2021`, and 24,000 hours of *VoxPopuli* :cite:`voxpopuli`, not fine-tuned. Originally published by the authors of *WavLM* :cite:`chen2022wavlm` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for the usage. zwav2vec2_xlsr_300m.ptha(XLS-R model with 300 million parameters, pre-trained on 436,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common`, *VoxLingua107* :cite:`valk2021voxlingua107`, *BABEL* :cite:`Gales2014SpeechRA`, and *VoxPopuli* :cite:`voxpopuli`) in 128 languages, not fine-tuned. Originally published by the authors of *XLS-R* :cite:`babu2021xls` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for usage details. zwav2vec2_xlsr_1b.ptha&XLS-R model with 1 billion parameters, pre-trained on 436,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common`, *VoxLingua107* :cite:`valk2021voxlingua107`, *BABEL* :cite:`Gales2014SpeechRA`, and *VoxPopuli* :cite:`voxpopuli`) in 128 languages, not fine-tuned. Originally published by the authors of *XLS-R* :cite:`babu2021xls` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for usage details. zwav2vec2_xlsr_2b.pthiia&XLS-R model with 2 billion parameters, pre-trained on 436,000 hours of unlabeled audio from multiple datasets ( *Multilingual LibriSpeech* :cite:`Pratap_2020`, *CommonVoice* :cite:`ardila2020common`, *VoxLingua107* :cite:`valk2021voxlingua107`, *BABEL* :cite:`Gales2014SpeechRA`, and *VoxPopuli* :cite:`voxpopuli`) in 128 languages, not fine-tuned. Originally published by the authors of *XLS-R* :cite:`babu2021xls` under MIT License and redistributed with the same license. [`License `__, `Source `__] Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2Bundle` for usage details. cseZdZdZGdddejZGdddejZde e e e e dfd fd d Z dd de edddZde e e ee efd ddZedddZedddZZS)Wav2Vec2FABundleuData class that bundles associated information to use pretrained :py:class:`~torchaudio.models.Wav2Vec2Model` for forced alignment. This class provides interfaces for instantiating the pretrained model along with the information necessary to retrieve pretrained weights and additional data to be used with the model. Torchaudio library instantiates objects of this class, each of which represents a different pretrained model. Client code should access pretrained models via these instances. Please see below for the usage and the available values. Example - Feature Extraction >>> import torchaudio >>> >>> bundle = torchaudio.pipelines.MMS_FA >>> >>> # Build the model and load pretrained weight. >>> model = bundle.get_model() Downloading: 100%|███████████████████████████████| 1.18G/1.18G [00:05<00:00, 216MB/s] >>> >>> # Resample audio to the expected sampling rate >>> waveform = torchaudio.functional.resample(waveform, sample_rate, bundle.sample_rate) >>> >>> # Estimate the probability of token distribution >>> emission, _ = model(waveform) >>> >>> # Generate frame-wise alignment >>> alignment, scores = torchaudio.functional.forced_align( >>> emission, targets, input_lengths, target_lengths, blank=0) >>> c@seZdZdZdS)zWav2Vec2FABundle.TokenizerzInterface of the tokenizerNr%r&r'r(rrrr TokenizersrZc@seZdZdZdS)zWav2Vec2FABundle.AlignerzInterface of the alignerNrYrrrrAlignersr[*r4.)starr6rcs$tj|d}|dkr|S||fS)aGet the labels corresponding to the feature dimension of emission. The first is blank token, and it is customizable. Args: star (str or None, optional): Change or disable star token. (default: ``"*"``) blank (str, optional): Change the blank token. (default: ``'-'``) Returns: Tuple[str, ...]: For models fine-tuned on ASR, returns the tuple of strings representing the output class labels. Example >>> from torchaudio.pipelines import MMS_FA as bundle >>> bundle.get_labels() ('-', 'a', 'i', 'e', 'n', 'o', 'u', 't', 's', 'r', 'm', 'k', 'l', 'd', 'g', 'h', 'y', 'b', 'p', 'w', 'c', 'v', 'j', 'z', 'f', "'", 'q', 'x', '*') >>> bundle.get_labels(star=None) ('-', 'a', 'i', 'e', 'n', 'o', 'u', 't', 's', 'r', 'm', 'k', 'l', 'd', 'g', 'h', 'y', 'b', 'p', 'w', 'c', 'v', 'j', 'z', 'f', "'", 'q', 'x') r5N)superr7)rr]r6labels __class__rrr7szWav2Vec2FABundle.get_labelsTNr) with_starrcCsLt|j|j}t|j||j}||tj||j d|d}| |S)a'Construct the model and load the pretrained weight. The weight file is downloaded from the internet and cached with :func:`torch.hub.load_state_dict_from_url` Args: with_star (bool, optional): If enabled, the last dimension of output layer is extended by one, which corresponds to `star` token. dl_kwargs (dictionary of keyword arguments): Passed to :func:`torch.hub.load_state_dict_from_url`. Returns: Variation of :py:class:`~torchaudio.models.Wav2Vec2Model`. .. note:: The model created with this method returns probability in log-domain, (i.e. :py:func:`torch.nn.functional.log_softmax` is applied), whereas the other Wav2Vec2 models returns logit. T)rZapply_log_softmaxZ append_star) r rrr rr r3rr rr!)rrbrr"r#rrrr$5s zWav2Vec2FABundle.get_modelcCsddt|j||dDS)aMGet the mapping from token to index (in emission feature dim) Args: star (str or None, optional): Change or disable star token. (default: ``"*"``) blank (str, optional): Change the blank token. (default: ``'-'``) Returns: Tuple[str, ...]: For models fine-tuned on ASR, returns the tuple of strings representing the output class labels. Example >>> from torchaudio.pipelines import MMS_FA as bundle >>> bundle.get_dict() {'-': 0, 'a': 1, 'i': 2, 'e': 3, 'n': 4, 'o': 5, 'u': 6, 't': 7, 's': 8, 'r': 9, 'm': 10, 'k': 11, 'l': 12, 'd': 13, 'g': 14, 'h': 15, 'y': 16, 'b': 17, 'p': 18, 'w': 19, 'c': 20, 'v': 21, 'j': 22, 'z': 23, 'f': 24, "'": 25, 'q': 26, 'x': 27, '*': 28} >>> bundle.get_dict(star=None) {'-': 0, 'a': 1, 'i': 2, 'e': 3, 'n': 4, 'o': 5, 'u': 6, 't': 7, 's': 8, 'r': 9, 'm': 10, 'k': 11, 'l': 12, 'd': 13, 'g': 14, 'h': 15, 'y': 16, 'b': 17, 'p': 18, 'w': 19, 'c': 20, 'v': 21, 'j': 22, 'z': 23, 'f': 24, "'": 25, 'q': 26, 'x': 27} cSsi|]\}}||qSrr).0ikrrr esz-Wav2Vec2FABundle.get_dict..)r]r6) enumerater7)rr]r6rrrget_dictRszWav2Vec2FABundle.get_dictrcCst|S)zIInstantiate a Tokenizer. Returns: Tokenizer )r rZrhrrrr get_tokenizergszWav2Vec2FABundle.get_tokenizercCs tjddS)zFInstantiate an Aligner. Returns: Aligner rr5)r r[rrrr get_aligneroszWav2Vec2FABundle.get_aligner)r\r4)T)r\r4)r%r&r'r(r Z ITokenizerrZZIAlignerr[rr)rr7r,rr$rr8rhrirj __classcell__rrr`rrXs"$ rXzQhttps://dl.fbaipublicfiles.com/mms/torchaudio/ctc_alignment_mling_uroman/model.pta Trained on 31K hours of data in 1,130 languages from *Scaling Speech Technology to 1,000+ Languages* :cite:`pratap2023scaling`. Published by the authors of *Scaling Speech Technology to 1,000+ Languages* :cite:`pratap2023scaling` under [`CC-BY-NC 4.0 License `__]. Please refer to :py:class:`torchaudio.pipelines.Wav2Vec2FABundle` for usage details. .. note:: Unlike other Wav2Vec2 bundles, this model does not have a token for word boundary (like `|`). This makes the post-processing of alignments slightly different. )6 dataclassesrtypingrrrrZtorch.nnrr r __all__r r.Z WAV2VEC2_BASEr(Z_get_en_labelsZWAV2VEC2_ASR_BASE_10MZWAV2VEC2_ASR_BASE_100HZWAV2VEC2_ASR_BASE_960HZWAV2VEC2_LARGEZWAV2VEC2_ASR_LARGE_10MZWAV2VEC2_ASR_LARGE_100HZWAV2VEC2_ASR_LARGE_960HZWAV2VEC2_LARGE_LV60KZWAV2VEC2_ASR_LARGE_LV60K_10MZWAV2VEC2_ASR_LARGE_LV60K_100HZWAV2VEC2_ASR_LARGE_LV60K_960HZWAV2VEC2_XLSR53Z HUBERT_BASEZ HUBERT_LARGEZ HUBERT_XLARGEZHUBERT_ASR_LARGEZHUBERT_ASR_XLARGEZ_get_de_labelsZVOXPOPULI_ASR_BASE_10K_DEZ_get_vp_en_labelsZVOXPOPULI_ASR_BASE_10K_ENZ_get_es_labelsZVOXPOPULI_ASR_BASE_10K_ESZ_get_fr_labelsZVOXPOPULI_ASR_BASE_10K_FRZ_get_it_labelsZVOXPOPULI_ASR_BASE_10K_ITZ WAVLM_BASEZWAVLM_BASE_PLUSZ WAVLM_LARGEZWAV2VEC2_XLSR_300MZWAV2VEC2_XLSR_1BZWAV2VEC2_XLSR_2BrXZ_get_mms_labelsZMMS_FArrrrs  WG    ! "  !     ! ! !     !  !  !             !  ! " " "  ! " "  "  "        !