U 9%e-+ @sbdZddlZddlmZddlmZmZddlmZm Z ddl Z ddl Z ddl mZddlmZddlmZdd lmZmZd d lmZd d lmZmZmZmZd d lmZeddddeddddeddddeddddeddddfZ eddd dZ!e"e#Z$ee%dged!d"d#hgd$gd%gd$gd$gd&d'd(dd#d'dd)d)d&d*d+Z&d,d-Z'd.d/Z(dS)0zhRCV1 dataset. The dataset page is available at http://jmlr.csail.mit.edu/papers/volume5/lewis04a/ N)GzipFile)makedirsremove)existsjoin)Bunch)shuffle) StrOptionsvalidate_params) get_data_home)RemoteFileMetadata _fetch_remote _pkl_filepath load_descr)load_svmlight_filesz.https://ndownloader.figshare.com/files/5976069Z@ed40f7e418d10484091b059703eeb95ae3199fe042891dcec4be6696b9968374z lyrl2004_vectors_test_pt0.dat.gz)urlZchecksumfilenamez.https://ndownloader.figshare.com/files/5976066Z@87700668ae45d45d5ca1ef6ae9bd81ab0f5ec88cc95dcef9ae7838f727a13aa6z lyrl2004_vectors_test_pt1.dat.gzz.https://ndownloader.figshare.com/files/5976063Z@48143ac703cbe33299f7ae9f4995db49a258690f60e5debbff8995c34841c7f5z lyrl2004_vectors_test_pt2.dat.gzz.https://ndownloader.figshare.com/files/5976060Z@dfcb0d658311481523c6e6ca0c3f5a3e1d3d12cde5d7a8ce629a9006ec7dbb39z lyrl2004_vectors_test_pt3.dat.gzz.https://ndownloader.figshare.com/files/5976057Z@5468f656d0ba7a83afc7ad44841cf9a53048a5c083eedc005dcdb5cc768924aezlyrl2004_vectors_train.dat.gzz.https://ndownloader.figshare.com/files/5976048Z@2a98e5e5d8b770bded93afc8930d88299474317fe14181aee1466cc754d0d1c1zrcv1v2.topics.qrels.gztraintestallboolean random_state) data_homesubsetdownload_if_missingrr return_X_yT)Zprefer_skip_nested_validationFc' Csd}d}d}d} t|d}t|d} |r8t| s8t| t| d} t| d} t| d } t| d }|rbt| rxt| sbg}tD]0}td |jt || d }| t |d qt ||d}t |d|d|d|d|dg}t|d|d|d|d|df}|jtjdd}tj|| ddtj|| dd|D]}|t|jqFnt| }t| }|rt| rt|std tjt t| d }d}d}d}tj||ftjd}tj|tjd}i}t |dd}|D]~}|d d!}t |dkr|\}} }!||kr8|d7}|||<t!| } | |kr^| }|d7}| ||<d||||f<qW5QRXt|t"||}"||"d"d"f}tj#|t$d}#|%D]}$|$|#||$<qt&|#}%|#|%}#t '|d"d"|%f}tj|| ddtj|#|ddnt| }t|}#|d#kr2n|d$krr|d"| d"d"f}|d"| d"d"f}|d"| }nL|d%kr|| d"d"d"f}|| d"d"d"f}|| d"}n t(d&||rt)||||d'\}}}t*d(}&|r||fSt+||||#|&d)S)*a| Load the RCV1 multilabel dataset (classification). Download it if necessary. Version: RCV1-v2, vectors, full sets, topics multilabels. ================= ===================== Classes 103 Samples total 804414 Dimensionality 47236 Features real, between 0 and 1 ================= ===================== Read more in the :ref:`User Guide `. .. versionadded:: 0.17 Parameters ---------- data_home : str, default=None Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. subset : {'train', 'test', 'all'}, default='all' Select the dataset to load: 'train' for the training set (23149 samples), 'test' for the test set (781265 samples), 'all' for both, with the training samples first if shuffle is False. This follows the official LYRL2004 chronological split. download_if_missing : bool, default=True If False, raise an OSError if the data is not locally available instead of trying to download the data from the source site. random_state : int, RandomState instance or None, default=None Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. shuffle : bool, default=False Whether to shuffle dataset. return_X_y : bool, default=False If True, returns ``(dataset.data, dataset.target)`` instead of a Bunch object. See below for more information about the `dataset.data` and `dataset.target` object. .. versionadded:: 0.20 Returns ------- dataset : :class:`~sklearn.utils.Bunch` Dictionary-like object. Returned only if `return_X_y` is False. `dataset` has the following attributes: - data : sparse matrix of shape (804414, 47236), dtype=np.float64 The array has 0.16% of non zero values. Will be of CSR format. - target : sparse matrix of shape (804414, 103), dtype=np.uint8 Each sample has a value of 1 in its categories, and 0 in others. The array has 3.15% of non zero values. Will be of CSR format. - sample_id : ndarray of shape (804414,), dtype=np.uint32, Identification number of each sample, as ordered in dataset.data. - target_names : ndarray of shape (103,), dtype=object Names of each target (RCV1 topics), as ordered in dataset.target. - DESCR : str Description of the RCV1 dataset. (data, target) : tuple A tuple consisting of `dataset.data` and `dataset.target`, as described above. Returned only if `return_X_y` is True. .. versionadded:: 0.20 i>F igimZ)rZRCV1z samples.pklz sample_id.pklzsample_topics.pklztopics_names.pklzDownloading %s)dirname)r)Z n_featuresrr r F)copy)compressZdtyperb)rmodeascii NrrrzLUnknown subset parameter. Got '%s' instead of one of ('all', 'train', test'))rzrcv1.rst)datatarget sample_idZ target_namesZDESCR),r rrrr XY_METADATAloggerinforrappendrrspZvstackZtocsrnpZhstackZastypeZuint32joblibdumpclosernameloadTOPICS_METADATAzerosZuint8int32decodesplitlenint_find_permutationemptyobjectkeysargsortZ csr_matrix ValueErrorshuffle_rr)'rrrrr rZ N_SAMPLESZ N_FEATURESZ N_CATEGORIESZN_TRAINZrcv1_dirZ samples_pathZsample_id_pathZsample_topics_pathZ topics_pathfilesZeach file_pathZXyXr1fZtopics_archive_pathZn_catZn_docZ doc_previousyZ sample_id_bisZcategory_nameslineZline_componentscatdoc_Z permutation categorieskorderZfdescrrWU/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/sklearn/datasets/_rcv1.py fetch_rcv1Ks\        ,(              rYcCs8|j}tj|tjd}tj|tjd}t||||S)zInverse permutation p.r*)sizer7r>r?Zarangeput)pnsirWrWrX_inverse_permutation$s r`cCs$t|}t|}t|}||S)z!Find the permutation from a to b.)r7rHr`)abtuZu_rWrWrXrD-s  rD))__doc__logginggziprosrros.pathrrr8numpyr7Z scipy.sparsesparser6utilsrr rJZutils._param_validationr r r _baserrrrZ_svmlight_format_iorr2r= getLogger__name__r3strrYr`rDrWrWrWrXs         O