U sVc@slddlmZddlmZddlmZddlmZddlm Z ddl m Z erXddl m Z Gdd d e Zd S) ) annotations) TYPE_CHECKING) ReadBuffer)import_optional_dependency) is_integer) ParserBase) DataFramecsVeZdZdZdddfdd ZddZd d Zd d d d dZd dddZZ S)ArrowParserWrapperz7 Wrapper for the pyarrow engine for read_csv() zReadBuffer[bytes]None)srcreturnc s$t|||_||_|dS)N)super__init__kwdsr _parse_kwds)selfr r __class__J/tmp/pip-unpacked-wheel-xj8nt62q/pandas/io/parsers/arrow_parser_wrapper.pyrs zArrowParserWrapper.__init__cCsf|jd}|dkrdn||_||jd\|_|_|jd}t|trRtdt |jd|_ dS)z? Validates keywords before passing to pyarrow. encodingNzutf-8usecols na_valuesz?The pyarrow engine doesn't support passing a dict for na_values) rgetrZ_validate_usecols_argrZ usecols_dtype isinstancedict ValueErrorlistr)rrrrrrrs    zArrowParserWrapper._parse_kwdscCsddddd}|D]4\}}||jkr|j|dk r|j||j|<qdd|jD|_d d|jD|_|jdk|jdk r|jn|jd d |_dS) z: Rename some arguments to pass to pyarrow include_columns null_values escape_charignore_empty_lines)rr escapecharZskip_blank_linesNcSs&i|]\}}|dk r|dkr||qS)N) delimiter quote_charr r!r.0Z option_nameZ option_valuerrr ;sz;ArrowParserWrapper._get_pyarrow_options..cSs&i|]\}}|dk r|dkr||qS)N)rrZ true_valuesZ false_valuesrr%rrrr'BsZskiprows)Zautogenerate_column_namesZ skip_rows)itemsrrpop parse_optionsconvert_optionsheader read_options)rmappingZ pandas_nameZ pyarrow_namerrr_get_pyarrow_options-s&z'ArrowParserWrapper._get_pyarrow_optionsr)framer c st|j}d}jdkrjdkrZjdk rFfddt|D_njdkrZt|_tj|krtt|tjj_d}j|_|j|\|_}jdk r4t jD]<\}}t |r|j|j|<q||jkrt d|dq|j jdddjdkr4|s4dgt|j j|j _jd dk rz|jd }Wn,tk r}z t |W5d}~XYnX|S) z Processes data read in based on kwargs. Parameters ---------- frame: DataFrame The DataFrame to process. Returns ------- DataFrame The processed DataFrame. TNcsg|]}j|qSr)prefix)r&irrr csz7ArrowParserWrapper._finalize_output..FzIndex z invalid)ZdropZinplaceZdtype)lencolumnsr,namesr1rangerZ_do_date_conversionsZ index_col enumeraterrZ set_indexindexrrZastype TypeError)rr0Znum_colsZmulti_index_namedr2itemerr3r_finalize_outputPs8        z#ArrowParserWrapper._finalize_output)r cCsTtd}||j|j|jf|j|jf|j|jf|j d}| }| |S)z Reads the contents of a CSV file into a DataFrame and processes it according to the kwargs passed in the constructor. Returns ------- DataFrame The DataFrame created from the CSV file. z pyarrow.csv)r-r*r+) rr/Zread_csvr Z ReadOptionsr-Z ParseOptionsr*ZConvertOptionsr+Z to_pandasr>)rZ pyarrow_csvtabler0rrrreads    zArrowParserWrapper.read) __name__ __module__ __qualname____doc__rrr/r>r@ __classcell__rrrrr s #5r N) __future__rtypingrZpandas._typingrZpandas.compat._optionalrZpandas.core.dtypes.inferencerZpandas.io.parsers.base_parserrZpandasrr rrrrs