U ,:%e-@s ddlZddlmZddlmZmZmZddlmZddl m Z ddl m Z ddl mZmZdZd d d d d gddddgddddddZdZdZedddZeeeeedddZeed d!d"Zeed#d$d%ZGd&d'd'e ZGd(d)d)eZGd*d+d+eZdS),N)Path)ListTupleUnion)Tensor)Dataset)download_url_to_file) _extract_zip_load_waveformi>zvox1_dev_wav.zipzIhttps://thor.robots.ox.ac.uk/~vgg/data/voxceleb/vox1a/vox1_dev_wav_partaazIhttps://thor.robots.ox.ac.uk/~vgg/data/voxceleb/vox1a/vox1_dev_wav_partabzIhttps://thor.robots.ox.ac.uk/~vgg/data/voxceleb/vox1a/vox1_dev_wav_partaczIhttps://thor.robots.ox.ac.uk/~vgg/data/voxceleb/vox1a/vox1_dev_wav_partadZ@21ec6ca843659ebc2fdbe04b530baa4f191ad4b0971912672d92c158f32226a0Z@311d21e0c8cbf33573a4fce6c80e5a279d80736274b381c394319fc557159a04Z@92b64465f2b2a3dc0e4196ae8dd6828cbe9ddd1f089419a11e4cbfe2e1750df0Z@00e6190c770b27f27d2a3dd26ee15596b17066b715ac111906861a7d09a211a5) archive_nameurls checksumszvox1_test_wav.zipzGhttps://thor.robots.ox.ac.uk/~vgg/data/voxceleb/vox1a/vox1_test_wav.zipZ@8de57f347fe22b2c24526e9f444f689ecf5096fc2a92018cf420ff6b5b15eaea)r urlchecksumdevtestzBhttps://www.robots.ox.ac.uk/~vgg/data/voxceleb/meta/iden_split.txtzAhttps://www.robots.ox.ac.uk/~vgg/data/voxceleb/meta/veri_test.txt)rootc CsdD]}t|d}tj||}|dkrt|d}t|d}t|db}t||D]P\}}tj|tj|} t|| |dt| d} || W5QRXqXW5QRXn&t|d }t|d }t|||dt |qdS) Nrr rr r wb)Z hash_prefixrbrr) _ARCHIVE_CONFIGSospathjoinopenzipbasenamerwritereadr ) rarchiver archive_pathr r frr file_pathZf_splitr#\/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/torchaudio/datasets/voxceleb1.py_download_extract_wavs&s      &  r%)rr"subsetreturnc Csng}|dkrd}n|dkr d}nd}t|d2}|D]&}|\}}t||kr4||q4W5QRXt|S)Ntrainrr)rsplitintappendsorted) rr"r&f_listindexr!lineidrr#r#r$ _get_flist<s   r5)rveri_test_pathc CsFg}t|d.}|D]"}|\}}}||||fqW5QRX|S)Nr,)rr-r/)rr6r1r!r3labelZpath1Zpath2r#r#r$_get_paired_flistLs  r8)r" _ext_audiocCs8|ddd\}}}||d}d|||g}|S)N/-)r-replacer)r"r9 speaker_idZ youtube_idZ utterance_idfile_idr#r#r$ _get_file_idUs rAc@sZeZdZdZdZdeeefeddddZ e dd d Z e dd d Z e d ddZ dS) VoxCeleb1a"*VoxCeleb1* :cite:`nagrani2017voxceleb` dataset. Args: root (str or Path): Path to the directory where the dataset is found or downloaded. download (bool, optional): Whether to download the dataset if it is not found at root path. (Default: ``False``). z.wavFN)rdownloadr'cCsJt|}tj|d|_tj|jsF|s>td|jdt|dS)NZwavzDataset not found at z5. Please set `download=True` to download the dataset.)rfspathrr_pathisdir RuntimeErrorr%)selfrrCr#r#r$__init__gs  zVoxCeleb1.__init__)ncCstdSNNotImplementedErrorrHrJr#r#r$ get_metadatarszVoxCeleb1.get_metadatacCstdSrKrLrNr#r#r$ __getitem__uszVoxCeleb1.__getitem__r'cCstdSrKrLrHr#r#r$__len__xszVoxCeleb1.__len__)F)__name__ __module__ __qualname____doc__r9rstrrboolrIr.rOrPrSr#r#r#r$rB\s  rBcseZdZdZdedfeeefeeeddfdd Z e e ee e efdd d Z e e e e e efdd d Ze d ddZZS)VoxCeleb1Identificationu*VoxCeleb1* :cite:`nagrani2017voxceleb` dataset for speaker identification task. Each data sample contains the waveform, sample rate, speaker id, and the file id. Args: root (str or Path): Path to the directory where the dataset is found or downloaded. subset (str, optional): Subset of the dataset to use. Options: ["train", "dev", "test"]. (Default: ``"train"``) meta_url (str, optional): The url of meta file that contains the list of subset labels and file paths. The format of each row is ``subset file_path". For example: ``1 id10006/nLEBBc9oIFs/00003.wav``. ``1``, ``2``, ``3`` mean ``train``, ``dev``, and ``test`` subest, respectively. (Default: ``"https://www.robots.ox.ac.uk/~vgg/data/voxceleb/meta/iden_split.txt"``) download (bool, optional): Whether to download the dataset if it is not found at root path. (Default: ``False``). Note: The file structure of `VoxCeleb1Identification` dataset is as follows: └─ root/ └─ wav/ └─ speaker_id folders Users who pre-downloaded the ``"vox1_dev_wav.zip"`` and ``"vox1_test_wav.zip"`` files need to move the extracted files into the same ``root`` directory. r(FN)rr&meta_urlrCr'cs^t|||dkrtdtj|tj|}tj|sJt||t |j |||_ dS)N)r(rrz0`subset` must be one of ['train', 'dev', 'test']) superrI ValueErrorrrrrexistsrr5rE_flist)rHrr&r[rCmeta_list_path __class__r#r$rIs  z VoxCeleb1Identification.__init__rJr'cCs@|j|}t||j}|dd}t|dd}|t||fS)aGet metadata for the n-th sample from the dataset. Returns filepath instead of waveform, but otherwise returns the same fields as :py:func:`__getitem__`. Args: n (int): The index of the sample Returns: Tuple of the following items; str: Path to audio int: Sample rate int: Speaker ID str: File ID r=rr+N)r_rAr9r-r. SAMPLE_RATE)rHrJr"r@r?r#r#r$rOs   z$VoxCeleb1Identification.get_metadatacCs2||}t|j|d|d}|f|ddS)adLoad the n-th sample from the dataset. Args: n (int): The index of the sample to be loaded Returns: Tuple of the following items; Tensor: Waveform int: Sample rate int: Speaker ID str: File ID rr)NrOr rE)rHrJmetadataZwaveformr#r#r$rPs z#VoxCeleb1Identification.__getitem__rQcCs t|jSrKlenr_rRr#r#r$rSszVoxCeleb1Identification.__len__)rTrUrVrW_IDEN_SPLIT_URLrrXrrYrIr.rrOrrPrS __classcell__r#r#rar$rZ|s  rZcseZdZdZedfeeefeeddfdd Z e e eee e eefddd Z e e e e e e eefdd d Ze d d dZZS)VoxCeleb1Verificationu*VoxCeleb1* :cite:`nagrani2017voxceleb` dataset for speaker verification task. Each data sample contains a pair of waveforms, sample rate, the label indicating if they are from the same speaker, and the file ids. Args: root (str or Path): Path to the directory where the dataset is found or downloaded. meta_url (str, optional): The url of meta file that contains a list of utterance pairs and the corresponding labels. The format of each row is ``label file_path1 file_path2". For example: ``1 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00008.wav``. ``1`` means the two utterances are from the same speaker, ``0`` means not. (Default: ``"https://www.robots.ox.ac.uk/~vgg/data/voxceleb/meta/veri_test.txt"``) download (bool, optional): Whether to download the dataset if it is not found at root path. (Default: ``False``). Note: The file structure of `VoxCeleb1Verification` dataset is as follows: └─ root/ └─ wav/ └─ speaker_id folders Users who pre-downloaded the ``"vox1_dev_wav.zip"`` and ``"vox1_test_wav.zip"`` files need to move the extracted files into the same ``root`` directory. FN)rr[rCr'csLt||tj|tj|}tj|s:t||t|j ||_ dSrK) r\rIrrrrr^rr8rEr_)rHrr[rCr`rar#r$rIs   zVoxCeleb1Verification.__init__rccCs@|j|\}}}t|}t||j}t||j}||t|||fS)agGet metadata for the n-th sample from the dataset. Returns filepaths instead of waveforms, but otherwise returns the same fields as :py:func:`__getitem__`. Args: n (int): The index of the sample Returns: Tuple of the following items; str: Path to audio file of speaker 1 str: Path to audio file of speaker 2 int: Sample rate int: Label str: File ID of speaker 1 str: File ID of speaker 2 )r_r.rAr9rd)rHrJr7Zfile_path_spk1Zfile_path_spk2Z file_id_spk1Z file_id_spk2r#r#r$rOs   z"VoxCeleb1Verification.get_metadatacCsJ||}t|j|d|d}t|j|d|d}||f|ddS)aLoad the n-th sample from the dataset. Args: n (int): The index of the sample to be loaded. Returns: Tuple of the following items; Tensor: Waveform of speaker 1 Tensor: Waveform of speaker 2 int: Sample rate int: Label str: File ID of speaker 1 str: File ID of speaker 2 rr*r)Nre)rHrJrfZ waveform_spk1Z waveform_spk2r#r#r$rPs z!VoxCeleb1Verification.__getitem__rQcCs t|jSrKrgrRr#r#r$rS4szVoxCeleb1Verification.__len__)rTrUrVrW_VERI_TEST_URLrrXrrYrIr.rrOrrPrSrjr#r#rar$rks &  rk)rpathlibrtypingrrrZtorchrZtorch.utils.datarZtorchaudio._internalrZtorchaudio.datasets.utilsr r rdrrirlrXr%r5r8rArBrZrkr#r#r#r$sB      [