U sVchw @sdZddlmZddlmZmZmZmZmZm Z m Z ddl Z ddl Z ddlmZddlmZddlmZddlmZmZmZmZmZddlmZdd lmZdd lmZm Z!dd l"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*dd l+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2dd l3m4Z4m5Z5ddl6m7Z7m8Z8m9Z9m:Z:m;Z;ddlm?m@ZAer\ddlBmCZCmDZDmEZEdMdddddddZFe dNdddddddZGe dOdddd dd!dZGdPdddd dd#dZGd$d%ZHd&d'd(d)d*ZIdQd"d+d,d-ddddd.d/d0ZJd1d'd2d3d4ZKd"d+dd-d,ddd5d6d7ZLd'd8dd'd9d:d;ZMdd,dd<d=d>ZNd?d-dddd@dAdBZOdCdd(dDdEZPddddd"d"eQfdCdFddGdddHdIdJdKdLZRdS)Rz Constructor functions intended to be shared by pd.array, Series.__init__, and Index.__new__. These should not depend on core.internals. ) annotations) TYPE_CHECKINGAnyOptionalSequenceUnioncastoverloadN)lib)Period) AnyArrayLike ArrayLikeDtypeDtypeObjT)IntCastingNaNError)find_stack_level)ExtensionDtype _registry)"construct_1d_arraylike_from_scalar'construct_1d_object_array_from_listlikemaybe_cast_to_datetimemaybe_cast_to_integer_arraymaybe_convert_platformmaybe_infer_to_datetimelike maybe_upcastsanitize_to_nanoseconds)is_datetime64_ns_dtypeis_extension_array_dtypeis_float_dtypeis_integer_dtype is_list_likeis_object_dtypeis_timedelta64_ns_dtype)DatetimeTZDtype PandasDtype)ABCExtensionArrayABCIndexABCPandasArray ABCRangeIndex ABCSeries)isna)ExtensionArrayIndexSeriesTzSequence[object] | AnyArrayLikez Dtype | Noneboolr,)datadtypecopyreturncCs,ddlm}m}m}m}m}m}m} m} m } ddl m } t |rVd|d} t| |dkrtt|tt|frt|j}t|dd}t|trt|p|}t|rtt|}|j|||d S|dkrt j|dd }|d krttttt t!f|}| j||d S|d kr|||d S|"drTz|j||d WStk rPYnXn|"drn| j||d S|dkr| j||d S|dkr|j||d S|dkrt#|ddt$j%kr|j||d S|dkr|j||d St&|r|j|||d St'|r| j|||d S| j|||d S)a Create an array. Parameters ---------- data : Sequence of objects The scalars inside `data` should be instances of the scalar type for `dtype`. It's expected that `data` represents a 1-dimensional array of data. When `data` is an Index or Series, the underlying array will be extracted from `data`. dtype : str, np.dtype, or ExtensionDtype, optional The dtype to use for the array. This may be a NumPy dtype or an extension type registered with pandas using :meth:`pandas.api.extensions.register_extension_dtype`. If not specified, there are two possibilities: 1. When `data` is a :class:`Series`, :class:`Index`, or :class:`ExtensionArray`, the `dtype` will be taken from the data. 2. Otherwise, pandas will attempt to infer the `dtype` from the data. Note that when `data` is a NumPy array, ``data.dtype`` is *not* used for inferring the array type. This is because NumPy cannot represent all the types of data that can be held in extension arrays. Currently, pandas will infer an extension dtype for sequences of ============================== ======================================= Scalar Type Array Type ============================== ======================================= :class:`pandas.Interval` :class:`pandas.arrays.IntervalArray` :class:`pandas.Period` :class:`pandas.arrays.PeriodArray` :class:`datetime.datetime` :class:`pandas.arrays.DatetimeArray` :class:`datetime.timedelta` :class:`pandas.arrays.TimedeltaArray` :class:`int` :class:`pandas.arrays.IntegerArray` :class:`float` :class:`pandas.arrays.FloatingArray` :class:`str` :class:`pandas.arrays.StringArray` or :class:`pandas.arrays.ArrowStringArray` :class:`bool` :class:`pandas.arrays.BooleanArray` ============================== ======================================= The ExtensionArray created when the scalar type is :class:`str` is determined by ``pd.options.mode.string_storage`` if the dtype is not explicitly given. For all other cases, NumPy's usual inference rules will be used. .. versionchanged:: 1.0.0 Pandas infers nullable-integer dtype for integer data, string dtype for string data, and nullable-boolean dtype for boolean data. .. versionchanged:: 1.2.0 Pandas now also infers nullable-floating dtype for float-like input data copy : bool, default True Whether to copy the data, even if not necessary. Depending on the type of `data`, creating the new array may require copying data, even if ``copy=False``. Returns ------- ExtensionArray The newly created array. Raises ------ ValueError When `data` is not 1-dimensional. See Also -------- numpy.array : Construct a NumPy array. Series : Construct a pandas Series. Index : Construct a pandas Index. arrays.PandasArray : ExtensionArray wrapping a NumPy array. Series.array : Extract the array stored within a Series. Notes ----- Omitting the `dtype` argument means pandas will attempt to infer the best array type from the values in the data. As new array types are added by pandas and 3rd party libraries, the "best" array type may change. We recommend specifying `dtype` to ensure that 1. the correct array type for the data is returned 2. the returned array type doesn't change as new extension types are added by pandas and third-party libraries Additionally, if the underlying memory representation of the returned array matters, we recommend specifying the `dtype` as a concrete object rather than a string alias or allowing it to be inferred. For example, a future version of pandas or a 3rd-party library may include a dedicated ExtensionArray for string data. In this event, the following would no longer return a :class:`arrays.PandasArray` backed by a NumPy array. >>> pd.array(['a', 'b'], dtype=str) ['a', 'b'] Length: 2, dtype: str32 This would instead return the new ExtensionArray dedicated for string data. If you really need the new array to be backed by a NumPy array, specify that in the dtype. >>> pd.array(['a', 'b'], dtype=np.dtype(" ['a', 'b'] Length: 2, dtype: str32 Finally, Pandas has arrays that mostly overlap with NumPy * :class:`arrays.DatetimeArray` * :class:`arrays.TimedeltaArray` When data with a ``datetime64[ns]`` or ``timedelta64[ns]`` dtype is passed, pandas will always return a ``DatetimeArray`` or ``TimedeltaArray`` rather than a ``PandasArray``. This is for symmetry with the case of timezone-aware data, which NumPy does not natively support. >>> pd.array(['2015', '2016'], dtype='datetime64[ns]') ['2015-01-01 00:00:00', '2016-01-01 00:00:00'] Length: 2, dtype: datetime64[ns] >>> pd.array(["1H", "2H"], dtype='timedelta64[ns]') ['0 days 01:00:00', '0 days 02:00:00'] Length: 2, dtype: timedelta64[ns] Examples -------- If a dtype is not specified, pandas will infer the best dtype from the values. See the description of `dtype` for the types pandas infers for. >>> pd.array([1, 2]) [1, 2] Length: 2, dtype: Int64 >>> pd.array([1, 2, np.nan]) [1, 2, ] Length: 3, dtype: Int64 >>> pd.array([1.1, 2.2]) [1.1, 2.2] Length: 2, dtype: Float64 >>> pd.array(["a", None, "c"]) ['a', , 'c'] Length: 3, dtype: string >>> with pd.option_context("string_storage", "pyarrow"): ... arr = pd.array(["a", None, "c"]) ... >>> arr ['a', , 'c'] Length: 3, dtype: string >>> pd.array([pd.Period('2000', freq="D"), pd.Period("2000", freq="D")]) ['2000-01-01', '2000-01-01'] Length: 2, dtype: period[D] You can use the string alias for `dtype` >>> pd.array(['a', 'b', 'a'], dtype='category') ['a', 'b', 'a'] Categories (2, object): ['a', 'b'] Or specify the actual dtype >>> pd.array(['a', 'b', 'a'], ... dtype=pd.CategoricalDtype(['a', 'b', 'c'], ordered=True)) ['a', 'b', 'a'] Categories (3, object): ['a' < 'b' < 'c'] If pandas does not infer a dedicated extension type a :class:`arrays.PandasArray` is returned. >>> pd.array([1 + 1j, 3 + 2j]) [(1+1j), (3+2j)] Length: 2, dtype: complex128 As mentioned in the "Notes" section, new extension types may be added in the future (by pandas or 3rd party libraries), causing the return value to no longer be a :class:`arrays.PandasArray`. Specify the `dtype` as a NumPy dtype if you need to ensure there's no future change in behavior. >>> pd.array([1, 2], dtype=np.dtype("int32")) [1, 2] Length: 2, dtype: int32 `data` must be 1-dimensional. A ValueError is raised when the input has the wrong dimensionality. >>> pd.array(1) Traceback (most recent call last): ... ValueError: Cannot pass scalar '1' to 'pandas.array'. r) BooleanArray DatetimeArrayr, FloatingArray IntegerArray IntervalArray PandasArray PeriodArrayTimedeltaArray) StringDtypezCannot pass scalar 'z' to 'pandas.array'.NT) extract_numpyr1r2)ZskipnaZperiodr2intervaldatetime timedeltastringinteger)Zfloatingzmixed-integer-floatr1boolean)(pandas.core.arraysr4r5r,r6r7r8r9r:r;Zpandas.core.arrays.string_r<r is_scalar ValueError isinstancer*r'r1 extract_arraystrregistryfindrrrconstruct_array_type_from_sequenceZ infer_dtyperrrr r startswithgetattrnpZfloat16rr#)r0r1r2r4r5r,r6r7r8r9r:r;r<msgclsZinferred_dtypeZ period_datarU>> extract_array(pd.Series(['a', 'b', 'c'], dtype='category')) ['a', 'b', 'c'] Categories (3, object): ['a', 'b', 'c'] Other objects like lists, arrays, and DataFrames are just passed through. >>> extract_array([1, 2, 3]) [1, 2, 3] For an ndarray-backed Series / Index the ndarray is returned. >>> extract_array(pd.Series([1, 2, 3])) array([1, 2, 3]) To extract all the way down to the ndarray, pass ``extract_numpy=True``. >>> extract_array(pd.Series([1, 2, 3]), extract_numpy=True) array([1, 2, 3]) )rIr'r*r)Z_valuesr(Zto_numpyr[rUrUrVrJs- cCsTt|tjrP|jjdkr.ddlm}||S|jjdkrPddlm}||S|S)zS Wrap datetime64 and timedelta64 ndarrays in DatetimeArray/TimedeltaArray. Mr)r5m)r;) rIrRndarrayr1kindrFr5rOr;)arrr5r;rUrUrVensure_wrapped_if_datetimelikes       razma.MaskedArrayz np.ndarray)r0r3cCs@t|}|r4t|dd\}}||||<n|}|S)z? Convert numpy MaskedArray to ensure mask is softened. Tr?)maZ getmaskarrayanyrZ soften_maskr2)r0maskZ fill_valuerUrUrVsanitize_masked_arrays  reallow_2dz Index | NonezDtypeObj | None)indexr1r2raise_cast_failurergr3c Cs&t|tjrt|}t|tr$|j}t|ddd}t|tjrb|j dkrb|dkrV|j }t |}nt|t rxt|}d}t|s|dkrtdt|t||}|St|tjrt|tjr|j}|dk rt|j rt|rz*tjddt|||d}W5QRXWntk rDtjd ttd tj||d }Yn\tk r|stjd ttd ttj |}tj|||d YStj||d }YnXnt||||}n4t|t r|}|dk r|j!||d }n|r|"}nt|t#t$frt%dt&|j'dt(|dr0tj||d }nt)|}|dk sPt|dkrzt||||}WnZtk rt|rtj|dd }|j j*dkrt+|||d||dYSnYnXn(t,|}|j t-krttj|}t.|}t/|||||d}t|tjr"ttj |}t0||||}|S)a Sanitize input data to an ndarray or ExtensionArray, copy if specified, coerce to the dtype if specified. Parameters ---------- data : Any index : Index or None, default None dtype : np.dtype, ExtensionDtype, or None, default None copy : bool, default False raise_cast_failure : bool, default True allow_2d : bool, default False If False, raise if we have a 2D Arraylike. Returns ------- np.ndarray or ExtensionArray Notes ----- raise_cast_failure=False is only intended to be True when called from the DataFrame constructor, as the dtype keyword there may be interpreted as only applying to a subset of columns, see GH#24435. T)r=rYrNFz2index must be specified when data is not list-likeignore)invalidaIn a future version, passing float-dtype values containing NaN and an integer dtype will raise IntCastingNaNError (subclass of ValueError) instead of silently ignoring the passed dtype. To retain the old behavior, call Series(arr) or DataFrame(arr) without passing a dtype. stacklevelr?zIn a future version, passing float-dtype values and an integer dtype to DataFrame will retain floating dtype if they cannot be cast losslessly (matching Series behavior). To retain the old behavior, use DataFrame(data).astype(dtype)r>'z' type is unorderedZ __array__f)r2rirgrf)1rIrbZ MaskedArrayrer%Z numpy_dtyperJrRr^ndimr1r Zitem_from_zerodimrangerange_to_ndarrayr!rHrlenZmatrixArr Zerrstate _try_castrwarningswarn FutureWarningrrWrr&astyper2set frozenset TypeErrortype__name__hasattrlistr_sanitize_arrayrobjectr_sanitize_ndim_sanitize_str_dtypes)r0rhr1r2rirgsubarrZcastedrUrUrVrs!                  rrq)rngr3c Csztj|j|j|jdd}Wntk r|jdkr@|jdksT|jdkr|jdkrztj|j|j|jdd}Wqtk rtt|}YqXn tt|}YnX|S)z) Cast a range object to ndarray. Zint64r1rZuint64)rRZarangestartstopstep OverflowErrorrr)rr`rUrUrVrrs(rr)resultr1rhrgr3cCst|dddkrtdn|jdkr0t||}nr|jdkrt|tjrV|rN|Stdt|rt|trt j |t dd}| }|j ||d}nt j ||d}|S)z6 Ensure we have a 1-dimensional result array. rprz(result should be arraylike with ndim > 0zData must be 1-dimensionalrr)rQrHrp _maybe_repeatrIrRr^r"rcomZasarray_tuplesafer1rNrO)rr0r1rhrgrTrUrUrVrs      rznp.dtype | None)rr1r2r3cCsJt|jjtrFt|sFtt|s6tj ||dd}tj |t |d}|S)z= Ensure we have a dtype that is supported by pandas. Fr>) issubclassr1r}rKr rGrRallr+rWr)rr0r1r2rUrUrVrs  r)r`rhr3cCs:|dk r6dt|kr$t|kr6nn|t|}|S)zx If we have a length-1 array and an index describing how long we expect the result to be, repeat the array. Nr)rsrepeat)r`rhrUrUrVrs rzlist | np.ndarray)r`r1r2rir3c Cst|tj}|dkr|rZttj|}|jtkr:t||dSt|}||krV|rV|}|Stj |dd}|jtks||j dkr|St|Snt|t rt|t rt ||S|j}||||d}|St|r|st|}|St|j||dS|jdkrH|r(ttj|}|j} |jdkr2|}n t|f} tj|d|d| S|jd kr^t ||Sz*t|rvt||}ntj |||d}WnNttfk r|rn*t j!d |d t"t#d tj |t|d}YnX|S) a Convert input to numpy ndarray and optionally cast to a given dtype. Parameters ---------- arr : ndarray or list Excludes: ExtensionArray, Series, Index. dtype : np.dtype, ExtensionDtype or None copy : bool If False, don't copy the data if not needed. raise_cast_failure : bool If True, and if a dtype is specified, raise errors during casting. Otherwise an object array is returned. Returns ------- np.ndarray or ExtensionArray Nr?Frr>Ur)Zconvert_na_valuer2)r]r\zCould not cast to z, falling back to object. This behavior is deprecated. In a future version, when a dtype is passed to 'DataFrame', either all columns will be cast to that dtype, or a TypeError will be raised.rl)$rIrRr^rr1rrrr2rWsizerr$rrNrOr"rraryr_shaperpZravelrsr Zensure_string_arrayZreshaper rrHr|rvrwrxr) r`r1r2riZ is_ndarrayoutZvarrZ array_typerrrUrUrVrusf                    rurcCs.|dk}t|ot|d }|o$| }|p,|S)a Utility to check if a Series is instantiated with empty data, which does not contain dtype information. Parameters ---------- data : array-like, Iterable, dict, or scalar value Contains data stored in Series. Returns ------- bool Nr1)r!r)r0Zis_noneZis_list_like_without_dtypeZis_simple_emptyrUrUrV is_empty_data]s rzArrayLike | Index | Nonez str | Nonerr.)r0rhr1namer2fastpathdtype_if_emptyr3cCs4ddlm}t|r |dkr |}|||||||dS)ag Helper to pass an explicit dtype when instantiating an empty Series. This silences a DeprecationWarning described in GitHub-17261. Parameters ---------- data : Mirrored from Series.__init__ index : Mirrored from Series.__init__ dtype : Mirrored from Series.__init__ name : Mirrored from Series.__init__ copy : Mirrored from Series.__init__ fastpath : Mirrored from Series.__init__ dtype_if_empty : str, numpy.dtype, or ExtensionDtype This dtype will be passed explicitly if an empty Series will be instantiated. Returns ------- Series r)r.N)r0rhr1rr2r)Zpandas.core.seriesr.r)r0rhr1rr2rrr.rUrUrV!create_series_with_explicit_dtypeqs r)NT)..)..)FF)NFT)S__doc__ __future__rtypingrrrrrrr rvZnumpyrRZnumpy.marbZ pandas._libsr Zpandas._libs.tslibs.periodr Zpandas._typingr r rrrZ pandas.errorsrZpandas.util._exceptionsrZpandas.core.dtypes.baserrrLZpandas.core.dtypes.castrrrrrrrrZpandas.core.dtypes.commonrrrr r!r"r#Zpandas.core.dtypes.dtypesr$r%Zpandas.core.dtypes.genericr&r'r(r)r*Zpandas.core.dtypes.missingr+Zpandas.core.commoncorecommonrZpandasr,r-r.rWrJrarerrrrrrrurrrrUrUrUrVsv $      ( $   2>%% u