U -e0@sddlZddlmZmZddlmZddlmZmZm Z m Z m Z m Z ddl ZddlZddlmZddlmZddlmZmZdd lmZdd lmZmZerd d lmZd \ZZ Z!eGdddZ"dS)N) dataclassfield)BytesIO) TYPE_CHECKINGAnyClassVarDictOptionalUnion)config)DownloadConfig)xopen xsplitext) array_cast)no_op_if_value_is_nullstring_to_dict) FeatureType)FFFc @s0eZdZUdZdZeeed<dZe ed<dZ e ed<dZ ee ed<dZ ee ed <eeed Zeeed <edd d d Ze ed<ddZee eefedddZd eeee ee e dffedddZedee dffdddZeejej fej dddZ!ej ej dddZ"dS)!Audioa1Audio [`Feature`] to extract audio data from an audio file. Input: The Audio feature accepts as input: - A `str`: Absolute path to the audio file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the audio file to the archive file. - `bytes`: Bytes content of the audio file. This is useful for archived files with sequential access. - A `dict` with the keys: - `path`: String with relative path of the audio file to the archive file. - `array`: Array containing the audio sample - `sampling_rate`: Integer corresponding to the sampling rate of the audio sample. This is useful for archived files with sequential access. Args: sampling_rate (`int`, *optional*): Target sampling rate. If `None`, the native sampling rate is used. mono (`bool`, defaults to `True`): Whether to convert the audio signal to mono by averaging samples across channels. decode (`bool`, defaults to `True`): Whether to decode the audio data. If `False`, returns the underlying dictionary in the format `{"path": audio_path, "bytes": audio_bytes}`. Example: ```py >>> from datasets import load_dataset, Audio >>> ds = load_dataset("PolyAI/minds14", name="en-US", split="train") >>> ds = ds.cast_column("audio", Audio(sampling_rate=16000)) >>> ds[0]["audio"] {'array': array([ 2.3443763e-05, 2.1729663e-04, 2.2145823e-04, ..., 3.8356509e-05, -7.3497440e-06, -2.1754686e-05], dtype=float32), 'path': '/root/.cache/huggingface/datasets/downloads/extracted/f14948e0e84be638dd7943ac36518a4cf3324e8b7aa331c5ab11541518e9368c/en-US~JOINT_ACCOUNT/602ba55abb1e6d0fbce92065.wav', 'sampling_rate': 16000} ``` N sampling_rateTmonodecodeiddictdtypebytespathpa_typeF)defaultinitrepr_typecCs|jSN)r)selfr&X/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/datasets/features/audio.py__call__MszAudio.__call__)valuereturnc Csz ddl}Wn,tk r8}ztd|W5d}~XYnXt|trNd|dSt|trb|ddSd|krt}|j||d|ddd|ddS|d dk rnt j |d rn|d  d r\|ddkrt d |d rtj|d tjd tjd}n tj|d dddtjd}tt}|j|||ddd|ddSd|d dSnF|d dk s|d dk r|d |d dStd|ddS)zEncode example into a format for Arrow. Args: value (`str` or `dict`): Data passed as input to Audio feature. Returns: `dict` rNz;To support encoding audio data, please install 'soundfile'.rarrayrZwav)formatrZpcmzBTo use PCM files, please specify a 'sampling_rate' in Audio objectr)rihr)rmodezUAn audio sample should have one of 'path' or 'bytes' but they are missing or None in .) soundfile ImportError isinstancestrrrwritegetvaluegetosrisfileendswithKeyErrornpZ frombufferZint16ZastypeZfloat32Zmemmap ValueError)r%r)sferrbufferZ bytes_valuer&r&r'encode_examplePs8      " "    zAudio.encode_example)r)token_per_repo_idr*c Cs|jstd|ddk r.|dt|dfn |ddf\}}|dkr^|dkr^td|dzddl}ddl}Wn,tk r}ztd|W5d}~XYnX|dk rt|d d dnd}t j s|d krtd nt j s|d krtd |dkr|pi}| dd} | t jr(t jnt j} zt| | d} || } Wnttfk rfd} YnXt| d} t|d| d}||\}}W5QRXn||\}}|j}|jr||}|jr|j|kr|j|||jd}|j}|||dS)aDecode example audio file into audio data. Args: value (`dict`): A dictionary with keys: - `path`: String with relative audio file path. - `bytes`: Bytes of the audio file. token_per_repo_id (`dict`, *optional*): To access and decode audio files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`) Returns: `dict` zMDecoding is disabled for this feature. Please use Audio(decode=True) instead.rNrzJAn audio sample should have one of 'path' or 'bytes' but both are None in r0rzJTo support decoding audio files, please install 'librosa' and 'soundfile'.rZopuszDecoding 'opus' files requires system library 'libsndfile'>=1.0.31, You can try to update `soundfile` python library: `pip install "soundfile>=0.12.1"`. Zmp3zDecoding 'mp3' files requires system library 'libsndfile'>=1.1.0, You can try to update `soundfile` python library: `pip install "soundfile>=0.12.1"`. z::repo_id)tokenrb)download_config)Zorig_srZ target_sr)rr+r)r RuntimeErrorrr=librosar1r2rlowerr ZIS_OPUS_SUPPORTEDZIS_MP3_SUPPORTEDsplit startswithZ HF_ENDPOINTZHUB_DATASETS_URLZHUB_DATASETS_HFFS_URLrr;r rreadTrZto_monorZresample)r%r)rBrfilerIr>r?Z audio_format source_urlpatternrDrErGfr+rr&r&r'decode_examplesP0 $      zAudio.decode_exampler)r*cCs,ddlm}|jrtd|d|ddS)z[If in the decodable state, raise an error, otherwise flatten the feature into a dictionary.r)Valuez'Cannot flatten a decoded Audio feature.binarystringr)featuresrTrr=)r%rTr&r&r'flattens  z Audio.flatten)storager*cCsvtj|jrLtjdgt|td}tjj||gddg| d}ntj |jrtjdgt|t d}tjj||gddg| d}ntj |jr|j drtdd|D}ntj |jrj|jdd kr|d}ntjdgt|td}|jdd kr0|d}ntjdgt|t d}tjj||gddg| d}t||jS) aCast an Arrow array to the Audio arrow storage type. The Arrow types that can be converted to the Audio pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the audio bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter Args: storage (`Union[pa.StringArray, pa.StructArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Audio arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})` Ntyperrmaskr+cSs$g|]}|dk rt|ndqSr$)rrA.0xr&r&r' sz&Audio.cast_storage..r)patypes is_stringr[r+lenrU StructArray from_arraysis_nullZ is_binaryrVZ is_structZget_all_field_indices to_pylistZget_field_indexrrrr%rYZ bytes_arrayZ path_arrayr&r&r' cast_storages""   zAudio.cast_storagecstddtjfdd|Dtd}tjdd|dDtd}tjj||gddg| d }t ||j S) a8Embed audio files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. Returns: `pa.StructArray`: Array in the Audio arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. c Ss"t|d}|}W5QRX|S)NrF)rrM)rrRbytes_r&r&r' path_to_bytess z*Audio.embed_storage..path_to_bytescs8g|]0}|dk r0|ddkr(|dq2|dndqS)Nrrr&r^rmr&r'ra sz'Audio.embed_storage..rZcSs$g|]}|dk rtj|ndqSr$)r8rbasename)r_rr&r&r'rasrrr\) rrbr+rirUrrVrfrgrhrrrjr&rnr' embed_storages   zAudio.embed_storage)N)#__name__ __module__ __qualname____doc__rr int__annotations__rboolrrr4rrrbstructrUrVrrrr#r(r rrrArrSrXZ StringArrayrfrkrpr&r&r&r'rs& +  $2 H &r)#r8 dataclassesrriortypingrrrrr r numpyr<Zpyarrowrbr Zdownload.download_configr Z#download.streaming_download_managerrrtablerZutils.py_utilsrrrWrZ_ffmpeg_warnedZ_librosa_warnedZ_audioread_warnedrr&r&r&r's