U ,:%eS@sddlZddlmZddlmZmZmZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZd Zd Zd Zd gd gd dgdZeeeeeeeefdddZGddde ZdS)N)Path)ListTupleUnion)Tensor)Dataset)download_url_to_file)_get_librispeech_metadata) _extract_tarZlibrispeech_finetuningzIhttps://dl.fbaipublicfiles.com/librilight/data/librispeech_finetuning.tgzZ@5d1efdc777b548194d7e09ba89126e2188026df9fd57aa57eb14408d2b2342afz1h/0z1h/*Z9h)10minZ1hZ10h)pathfolders _ext_audioreturncs^tg}|D]8}fdd|d|D}|dd|D7}q|jddd|S)aGet the file names and the corresponding file paths without `speaker_id` and `chapter_id` directories. The format of path is like: {root}/{_ARCHIVE_NAME}/1h/[0-5]/[clean, other] or {root}/{_ARCHIVE_NAME}/9h/[clean, other] Args: path (Path): Root path to the dataset. folders (List[str]): Folders that contain the desired audio files. _ext_audio (str): Extension of audio files. Returns: List[Tuple[str, str]]: List of tuples where the first element is the relative path to the audio file. The format of relative path is like: 1h/[0-5]/[clean, other] or 9h/[clean, other] The second element is the file name without audio extension. csg|]}|qS) relative_to.0pr re/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/torchaudio/datasets/librilight_limited.py *sz&_get_fileids_paths..z/*/*/*/*cSs$g|]}t|jjjt|jfqSr)strparentstemrrrrr+scSs|d|dS)Nrr)xrrr,z$_get_fileids_paths..)key)rglobsort)r r rZ files_pathsfolderpathsrrr_get_fileids_pathss"r$c@sdeZdZdZdZdZdeeefee dddd Z e e e e ee e e fd d d Ze d ddZdS)LibriLightLimitedaSubset of Libri-light :cite:`librilight` dataset, which was used in HuBERT :cite:`hsu2021hubert` for supervised fine-tuning. Args: root (str or Path): Path to the directory where the dataset is found or downloaded. subset (str, optional): The subset to use. Options: [``"10min"``, ``"1h"``, ``"10h"``] (Default: ``"10min"``). download (bool, optional): Whether to download the dataset if it is not found at root path. (default: ``False``). z .trans.txtz.flacr FN)rootsubsetdownloadrcCs|tkr tdtd|t|}t|}tj|t|_tj|td}tj |js|spt dtj |st t |tdt|t|j||j|_dS)Nz`subset` must be one of z . Found: z.tgzz9Dataset not found. Please use `download=True` to download)Z hash_prefix) _SUBSET_MAP ValueErrorkeysosfspathr join _ARCHIVE_NAME_pathisdir RuntimeErrorisfiler_URL _CHECKSUMr r$r_fileids_paths)selfr&r'r(r archiverrr__init__?s  zLibriLightLimited.__init__)nrcCsT|j|\}}t||j||j|j}ttj |j|d\}}|f|ddS)aLoad the n-th sample from the dataset. Args: n (int): The index of the sample to be loaded Returns: Tuple of the following items; Tensor: Waveform int: Sample rate str: Transcript int: Speaker ID int: Chapter ID int: Utterance ID rrN) r6r r0r_ext_txt torchaudioloadr,r r.)r7r: file_pathZfileidmetadataZwaveform_rrr __getitem__TszLibriLightLimited.__getitem__)rcCs t|jS)N)lenr6)r7rrr__len__nszLibriLightLimited.__len__)r F)__name__ __module__ __qualname____doc__r;rrrrboolr9intrrrArCrrrrr%0s    r%)r,pathlibrtypingrrrr<ZtorchrZtorch.utils.datarZtorchaudio._internalrZtorchaudio.datasets.librispeechr Ztorchaudio.datasets.utilsr r/r4r5r)rr$r%rrrrs      $