U d@sUdZddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z m Z m Z mZmZmZddlmZmZmZmZddlmZddlmZmZddlmZmZmZmZm Z ddl!m"Z"dd l#m$Z$dd l%m&Z&dd l'm(Z(dd l)m*Z*dd l+m,Z,ddl-m.Z.ddl/m0Z0m1Z1ddl2m3Z3ddl4m5Z5ej6ej7ej8ej9ej:ej;dZGdddeZ?e?e>j@dZAe?eBd<e?e>jCdZDe?eBd<e?e>jEdZFe?eBd<eFZGe?eBd<eeeeHddd ZIeeHeJeKfeeed!d"d#ZLeGfee?ed$d%d&ZMeGfe e eHefe?ed'd(d)ZNeGfe eHefe?ed*d+d,ZOeed-d.d/ZPee?eeejQfd0d1d2ZReeSe?eeejQfd3d4d5ZTee?eeejQfd0d6d7ZUee?eeejQfd0d8d9ZVee?eejefd0d:d;ZWee,d-dd?ZYeed-d@dAZZee.d-dBdCZ[ee"d-dDdEZ\eed-dFdGZ]eeSd-dHdIZ^ee&d-dJdKZ_ee`d-dLdMZaee$d-dNdOZbee*d-dPdQZcee(d-dRdSZdeJeSe?ed3dTdUZeeGfee?ed$dVdWZfdS)XaOTools for using Python's :mod:`json` module with BSON documents. This module provides two helper methods `dumps` and `loads` that wrap the native :mod:`json` methods and provide explicit BSON conversion to and from JSON. :class:`~bson.json_util.JSONOptions` provides a way to control how JSON is emitted and parsed, with the default being the Relaxed Extended JSON format. :mod:`~bson.json_util` can also generate Canonical or legacy `Extended JSON`_ when :const:`CANONICAL_JSON_OPTIONS` or :const:`LEGACY_JSON_OPTIONS` is provided, respectively. .. _Extended JSON: https://github.com/mongodb/specifications/blob/master/source/extended-json.rst Example usage (deserialization): .. doctest:: >>> from bson.json_util import loads >>> loads( ... '[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$scope": {}, "$code": "function x() { return 1; }"}}, {"bin": {"$type": "80", "$binary": "AQIDBA=="}}]' ... ) [{'foo': [1, 2]}, {'bar': {'hello': 'world'}}, {'code': Code('function x() { return 1; }', {})}, {'bin': Binary(b'...', 128)}] Example usage with :const:`RELAXED_JSON_OPTIONS` (the default): .. doctest:: >>> from bson import Binary, Code >>> from bson.json_util import dumps >>> dumps( ... [ ... {"foo": [1, 2]}, ... {"bar": {"hello": "world"}}, ... {"code": Code("function x() { return 1; }")}, ... {"bin": Binary(b"")}, ... ] ... ) '[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$code": "function x() { return 1; }"}}, {"bin": {"$binary": {"base64": "AQIDBA==", "subType": "00"}}}]' Example usage (with :const:`CANONICAL_JSON_OPTIONS`): .. doctest:: >>> from bson import Binary, Code >>> from bson.json_util import dumps, CANONICAL_JSON_OPTIONS >>> dumps( ... [ ... {"foo": [1, 2]}, ... {"bar": {"hello": "world"}}, ... {"code": Code("function x() { return 1; }")}, ... {"bin": Binary(b"")}, ... ], ... json_options=CANONICAL_JSON_OPTIONS, ... ) '[{"foo": [{"$numberInt": "1"}, {"$numberInt": "2"}]}, {"bar": {"hello": "world"}}, {"code": {"$code": "function x() { return 1; }"}}, {"bin": {"$binary": {"base64": "AQIDBA==", "subType": "00"}}}]' Example usage (with :const:`LEGACY_JSON_OPTIONS`): .. doctest:: >>> from bson import Binary, Code >>> from bson.json_util import dumps, LEGACY_JSON_OPTIONS >>> dumps( ... [ ... {"foo": [1, 2]}, ... {"bar": {"hello": "world"}}, ... {"code": Code("function x() { return 1; }", {})}, ... {"bin": Binary(b"")}, ... ], ... json_options=LEGACY_JSON_OPTIONS, ... ) '[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$code": "function x() { return 1; }", "$scope": {}}}, {"bin": {"$binary": "AQIDBA==", "$type": "00"}}]' Alternatively, you can manually pass the `default` to :func:`json.dumps`. It won't handle :class:`~bson.binary.Binary` and :class:`~bson.code.Code` instances (as they are extended strings you can't provide custom defaults), but it will be faster as there is less recursion. .. note:: If your application does not need the flexibility offered by :class:`JSONOptions` and spends a large amount of time in the `json_util` module, look to `python-bsonjs `_ for a nice performance improvement. `python-bsonjs` is a fast BSON to MongoDB Extended JSON converter for Python built on top of `libbson `_. `python-bsonjs` works best with PyMongo when using :class:`~bson.raw_bson.RawBSONDocument`. N) AnyDictMappingOptionalSequenceTupleTypeUnioncast)ALL_UUID_SUBTYPES UUID_SUBTYPEBinaryUuidRepresentation)Code) CodecOptionsDatetimeConversion) EPOCH_AWARE DatetimeMS_datetime_to_millis_max_datetime_ms_millis_to_datetime)DBRef) Decimal128)Int64)MaxKey)MinKey)ObjectId)Regex)RE_TYPESON) Timestamp)utc)ilmsuxc@seZdZdZdZdZdS)DatetimeRepresentationrN)__name__ __module__ __qualname__LEGACY NUMBERLONGISO8601r1r12/tmp/pip-unpacked-wheel-oblwsawz/bson/json_util.pyr(s  r(c@seZdZdZdZdZdS)JSONModerr)r*N)r+r,r-r.RELAXED CANONICALr1r1r1r2r3s r3c seZdZUeed<eed<eed<eed<eedfdd Zdddej fe de ee ee eeeedd fd d Z e d fd d Zeeefd fdd ZeddddZZS) JSONOptions json_modestrict_number_longdatetime_representation strict_uuid)argskwargscstdS)a Encapsulates JSON options for :func:`dumps` and :func:`loads`. :Parameters: - `strict_number_long`: If ``True``, :class:`~bson.int64.Int64` objects are encoded to MongoDB Extended JSON's *Strict mode* type `NumberLong`, ie ``'{"$numberLong": "" }'``. Otherwise they will be encoded as an `int`. Defaults to ``False``. - `datetime_representation`: The representation to use when encoding instances of :class:`datetime.datetime`. Defaults to :const:`~DatetimeRepresentation.LEGACY`. - `strict_uuid`: If ``True``, :class:`uuid.UUID` object are encoded to MongoDB Extended JSON's *Strict mode* type `Binary`. Otherwise it will be encoded as ``'{"$uuid": "" }'``. Defaults to ``False``. - `json_mode`: The :class:`JSONMode` to use when encoding BSON types to Extended JSON. Defaults to :const:`~JSONMode.LEGACY`. - `document_class`: BSON documents returned by :func:`loads` will be decoded to an instance of this class. Must be a subclass of :class:`collections.MutableMapping`. Defaults to :class:`dict`. - `uuid_representation`: The :class:`~bson.binary.UuidRepresentation` to use when encoding and decoding instances of :class:`uuid.UUID`. Defaults to :const:`~bson.binary.UuidRepresentation.UNSPECIFIED`. - `tz_aware`: If ``True``, MongoDB Extended JSON's *Strict mode* type `Date` will be decoded to timezone aware instances of :class:`datetime.datetime`. Otherwise they will be naive. Defaults to ``False``. - `tzinfo`: A :class:`datetime.tzinfo` subclass that specifies the timezone from which :class:`~datetime.datetime` objects should be decoded. Defaults to :const:`~bson.tz_util.utc`. - `datetime_conversion`: Specifies how UTC datetimes should be decoded within BSON. Valid options include 'datetime_ms' to return as a DatetimeMS, 'datetime' to return as a datetime.datetime and raising a ValueError for out-of-range values, 'datetime_auto' to return DatetimeMS objects when the underlying datetime is out-of-range and 'datetime_clamp' to clamp to the minimum and maximum possible datetimes. Defaults to 'datetime'. See :ref:`handling-out-of-range-datetimes` for details. - `args`: arguments to :class:`~bson.codec_options.CodecOptions` - `kwargs`: arguments to :class:`~bson.codec_options.CodecOptions` .. seealso:: The specification for Relaxed and Canonical `Extended JSON`_. .. versionchanged:: 4.0 The default for `json_mode` was changed from :const:`JSONMode.LEGACY` to :const:`JSONMode.RELAXED`. The default for `uuid_representation` was changed from :const:`~bson.binary.UuidRepresentation.PYTHON_LEGACY` to :const:`~bson.binary.UuidRepresentation.UNSPECIFIED`. .. versionchanged:: 3.5 Accepts the optional parameter `json_mode`. .. versionchanged:: 4.0 Changed default value of `tz_aware` to False. N)super__init__)selfr;r< __class__r1r2r>s7zJSONOptions.__init__N)clsr8r9r:r7r;r<returncs||dd|d<|dr(|dt|d<|tjtjtjdfkrFtdttt j |f||}|t jt j t j fkr|td||_|jt j kr|rtd|dtjfkrtd|dkrtd d|_tj|_d |_n|jt j kr4|dkrtd |dtjfkr td |dkrtd d |_tj|_d |_nDd|_tj|_d|_|dk rX||_|dk rh||_|dk rx||_|S) Ntz_awareFtzinfoznJSONOptions.datetime_representation must be one of LEGACY, NUMBERLONG, or ISO8601 from DatetimeRepresentation.zQJSONOptions.json_mode must be one of LEGACY, RELAXED, or CANONICAL from JSONMode.z>> from bson.json_util import CANONICAL_JSON_OPTIONS >>> CANONICAL_JSON_OPTIONS.tz_aware True >>> json_options = CANONICAL_JSON_OPTIONS.with_options(tz_aware=False, tzinfo=None) >>> json_options.tz_aware False .. versionadded:: 3.12 rK)rLrFgetattrrMr6)r?r<optsoptr1r1r2 with_optionsys  zJSONOptions.with_options)r+r,r-int__annotations__boolrr>r3r4rrrHstrrJrrLrQ __classcell__r1r1r@r2r6s, ;C  r6)r7LEGACY_JSON_OPTIONSCANONICAL_JSON_OPTIONSRELAXED_JSON_OPTIONSDEFAULT_JSON_OPTIONS)objr;r<rCcOs$|dt}tjt||f||S)aaHelper function that wraps :func:`json.dumps`. Recursive function that handles all BSON types including :class:`~bson.binary.Binary` and :class:`~bson.code.Code`. :Parameters: - `json_options`: A :class:`JSONOptions` instance used to modify the encoding of MongoDB Extended JSON types. Defaults to :const:`DEFAULT_JSON_OPTIONS`. .. versionchanged:: 4.0 Now outputs MongoDB Relaxed Extended JSON by default (using :const:`DEFAULT_JSON_OPTIONS`). .. versionchanged:: 3.4 Accepts optional parameter `json_options`. See :class:`JSONOptions`. json_options)poprZjsondumps _json_convert)r[r;r<r\r1r1r2r_s r_)r%r;r<rCcs.|dtfdd|d<tj|f||S)aHelper function that wraps :func:`json.loads`. Automatically passes the object_hook for BSON type conversion. Raises ``TypeError``, ``ValueError``, ``KeyError``, or :exc:`~bson.errors.InvalidId` on invalid MongoDB Extended JSON. :Parameters: - `json_options`: A :class:`JSONOptions` instance used to modify the decoding of MongoDB Extended JSON types. Defaults to :const:`DEFAULT_JSON_OPTIONS`. .. versionchanged:: 4.0 Now loads :class:`datetime.datetime` instances as naive by default. To load timezone aware instances utilize the `json_options` parameter. See :ref:`tz_aware_default_change` for an example. .. versionchanged:: 3.5 Parses Relaxed and Canonical Extended JSON as well as PyMongo's legacy format. Now raises ``TypeError`` or ``ValueError`` when parsing JSON type wrappers with values of the wrong type or any extra keys. .. versionchanged:: 3.4 Accepts optional parameter `json_options`. See :class:`JSONOptions`. r\cs t|SN)object_pairs_hook)pairsr\r1r2zloads..rb)r]rZr^loads)r%r;r<r1rdr2rgs rg)r[r\rCcsxt|dr$tfdd|DSt|drNt|ttfsNfdd|DSz t|WStk rr|YSXdS)z]Recursive helper method that converts BSON types so they can be converted into json. itemsc3s |]\}}|t|fVqdSrar`).0kvrdr1r2 sz _json_convert..__iter__csg|]}t|qSr1ri)rjrlrdr1r2 sz!_json_convert..N)hasattrrrh isinstancerUbytesdefault TypeError)r[r\r1rdr2r`s  r`)rcr\rCcCst|||Sra) object_hookZdocument_class)rcr\r1r1r2rbsrb)dctr\rCcCsd|krt|St|dtrHd|krHt|dttdfrHt|Sd|krZt||Sd|krjt|Sd|krzt|Sd|krt |Sd |krd |krt ||St ||Sd |krt |Sd |krt ||Sd |krdSd|krt|Sd|kr|d}t|d|dSd|kr"t|Sd|kr4t|Sd|krFt|Sd|krXt|Sd|krjt|Sd|kr|t|S|S)N$oid$ref$id$db$date$regex$minKey$maxKey$binary$type$code$uuidz $undefined $numberLong $timestamptr"$numberDecimal $dbPointer$regularExpression$symbol $numberInt $numberDouble)_parse_canonical_oidrqrFrUtype_parse_canonical_dbref_parse_canonical_datetime_parse_legacy_regex_parse_canonical_minkey_parse_canonical_maxkey_parse_legacy_binary_parse_canonical_binary_parse_canonical_code_parse_legacy_uuid_parse_canonical_int64r _parse_canonical_decimal128_parse_canonical_dbpointer_parse_canonical_regex_parse_canonical_symbol_parse_canonical_int32_parse_canonical_double)rvr\Ztspr1r1r2rusZ           ru)docrCcCsJ|d}t|ttfs|Sd}|ddD]}|t|dO}q*t||S)Nr|r$options)rqrUrrrF _RE_OPT_TABLEr)rpatternflagsrPr1r1r2r+sr)rr\rCcCsht|dkrtd|t|dts6td||jtjkrVtt |dSt |dSdS)z*Decode a JSON legacy $uuid to Python UUID.r)zBad $uuid, extra field(s): rz$uuid must be a string: N) lenrtrqrUuuid_representationr UNSPECIFIEDr from_uuiduuidUUID)rr\r1r1r2r7s  r)datasubtyper\rCcCsn|tkrP|j}t||}|tjkr&|S|tkr6tj}n|tjkrFtj}||S|dkrdt t j |St||S)Nr) r rr rrr ZSTANDARDZ PYTHON_LEGACYZas_uuidr rr)rrr\rZ binary_valuer1r1r2_binary_or_uuidCs     rcCsht|dtrd|d|d<t|dd}|dkrJt|dddd}t|d}t|||S)Nr%02xlr)rqrRbase64 b64decodeencoder)rr\rrr1r1r2rXsrcCs|d}|d}|d}t|ts0td|t|trFt|dkrTtd|t|dkrntd|t|}t|t|d|S) NrrsubTypez!$binary base64 must be a string: r*z7$binary subType must be a string at most 2 characters: z=$binary must include only "base64" and "subType" components: r) rqrUrtrrrrrrR)rr\binaryZb64rrr1r1r2rbs  rc Cs^|d}t|dkr"td|t|trP|ddkrL|dd}d}n|ddkr~|d d kr~|dd}|dd}nT|d dkr|dd }|d d}n.|d dkr|dd }|d d}n|}d }|d }d}|dkrtt||dd}|d|}tj|dj |t d}|r|dkrt|dkrt|dd d \}} t|dt| d} nZt|dkrt|dddt|ddd} n"t|dkrt|ddd} |ddkr| d9} |tj | d}|j r*|jr||j}|jtjkr&t|S|S|j dd} |jtjkrLt| S| Stt||S)z3Decode a JSON datetime to python datetime.datetime.r{r)zBad $date, extra field(s): ZNi)+-:r.ri@B%Y-%m-%dT%H:%M:%S) microsecondrEri<r)secondsrE)rrtrqrUrfindrRfloatdatetimestrptimereplacer!split timedeltarDrE astimezoneZdatetime_conversionrZ DATETIME_MSrr) rr\ZdtmdtoffsetZ dot_indexrZawarehoursminutesZsecsZaware_tzinfo_noner1r1r2rqsb            *  rcCs&t|dkrtd|t|dS)z1Decode a JSON ObjectId to bson.objectid.ObjectId.r)zBad $oid, extra field(s): rw)rrtrrr1r1r2rs rcCs*|d}t|dkr"td|t|S)z&Decode a JSON symbol to Python string.rr)zBad $symbol, extra field(s): )rrtrU)rsymbolr1r1r2rs rcCs6|D]}|dkrtd|qt|d|ddS)z%Decode a JSON code to bson.code.Code.)r$scopezBad $code, extra field(s): rr)scope)rtrrF)rkeyr1r1r2rsrcCsl|d}t|dkr"td|t|dkrtkrPnnt||S|jtjkrnd tt|iSd d tt|iiS|jrt|trd t|iSt|tt frd }|j!t"j#@r|d7}|j!t"j$@r|d7}|j!t"j%@r|d7}|j!t"j&@r|d7}|j!t"j'@r$|d7}|j!t"j(@r:|d7}t|j)trP|j)}n |j)*d}|j+t,jkr~t-d|fd|fgSdt-d|fd|fgiSt|t.rddiSt|t/rddiSt|t0rdt-d|j1fd|j2fgiSt|t3r*|j4dkr d t|iSt-d t|fd!t|j4|fgSt|t5rDt6||j7|St|t8r\t6|d|St|t9j:r|j;rt5j<||j=d"}t6||j7|Sd#|j>iSt|t?rd$t|iSt|t@r|S|j+t,jAkrt|trd%|krd&krnn d't|iSd t|iS|j+t,jkrt|tBrtCD|r>d(d)iStCE|rd|dkrXd*nd+} d(| iS|j+t,jAkrd(ttF|iStGd,|dS)-Nrwrdr)rrrrz%ziz.%03drr{z{}{}{}rrrr"r#r$r%r&r'zutf-8r|rrrrr}r)r~rrrr)rrrilrrNaNInfinityz -Infinityz%r is not JSON serializable)HrqrrUrr`rrr9r(r0rErr!AssertionErrorr utcoffsetdaysr microsecondsstrftimerRrrIrr.rrrsZ as_datetimer8rrrrre IGNORECASELOCALE MULTILINEDOTALLUNICODEVERBOSErrr7r3rrrr timeincrrr rrrrrrr:rrhexrrTr5rmathisnanisinfreprrt) r[r\offZ tz_stringZmillisZfracsecsrrZbinvalZrepresentationr1r1r2rs4s                                rs)g__doc__rrr^rrrtypingrrrrrrrr r Z bson.binaryr r r rZ bson.coderZbson.codec_optionsrrZbson.datetime_msrrrrrZ bson.dbrefrZbson.decimal128rZ bson.int64rZ bson.max_keyrZ bson.min_keyrZ bson.objectidrZ bson.regexrZbson.sonrrZbson.timestampr Z bson.tz_utilr!ILMSUXrr(r3r6r.rWrSr5rXr4rYrZrUr_rr bytearrayrgr`rbrurrrrRrrrrrrrrrrrrrrrrrrrsr1r1r1r2sX,           $+1  0    B