U M8ÚcâOã@södZddlmZddlmZddlmZdZdZdZ Gdd „d e ƒZ Gd d „d e ƒZ Gd d „d e ƒZ Gdd„de ƒZGdd„de ƒZGdd„de ƒZGdd„dƒZd#dd„ZGdd„dƒZGdd„dƒZGdd„dƒZGdd„dƒZGd d!„d!ƒZd"S)$zBinary Event Stream Decoding é)Úcrc32)Úunpack)ÚEventStreamErroré iic@seZdZdZdS)Ú ParserErrorz,Base binary flow encoding parsing exception.N)Ú__name__Ú __module__Ú __qualname__Ú__doc__©r r ú8/tmp/pip-unpacked-wheel-ozje0y8b/botocore/eventstream.pyrsrcs eZdZdZ‡fdd„Z‡ZS)ÚDuplicateHeaderz$Duplicate header found in the event.csd|}tƒ |¡dS)NzDuplicate header present: "%s"©ÚsuperÚ__init__)ÚselfÚheaderÚmessage©Ú __class__r r r#szDuplicateHeader.__init__©rrr r rÚ __classcell__r r rr r sr cs eZdZdZ‡fdd„Z‡ZS)ÚInvalidHeadersLengthz*Headers length is longer than the maximum.csd |t¡}tƒ |¡dS)Nz.Header length of {} exceeded the maximum of {})ÚformatÚ_MAX_HEADERS_LENGTHrr©rÚlengthrrr r r+s þzInvalidHeadersLength.__init__rr r rr r(srcs eZdZdZ‡fdd„Z‡ZS)ÚInvalidPayloadLengthz*Payload length is longer than the maximum.csd |t¡}tƒ |¡dS)Nz/Payload length of {} exceeded the maximum of {})rÚ_MAX_PAYLOAD_LENGTHrrrrr r r6s þzInvalidPayloadLength.__init__rr r rr r3srcs eZdZdZ‡fdd„Z‡ZS)ÚChecksumMismatchz8Calculated checksum did not match the expected checksum.csd ||¡}tƒ |¡dS)Nz9Checksum mismatch: expected 0x{:08x}, calculated 0x{:08x})rrr)rÚexpectedZ calculatedrrr r rAs þÿzChecksumMismatch.__init__rr r rr r>srcs eZdZdZ‡fdd„Z‡ZS)ÚNoInitialResponseErrorzÊAn event of type initial-response was not received. This exception is raised when the event stream produced no events or the first event in the stream was not of the initial-response type. csd}tƒ |¡dS)Nz0First event was not of the initial-response typer)rrrr r rRszNoInitialResponseError.__init__rr r rr r!Ksr!c@sÐeZdZdZdZdZdZdZdZdZ dZ d Z eeed œZ e d d „ƒZe d d„ƒZe dd„ƒZe dd„ƒZe dd„ƒZe dd„ƒZe dd„ƒZe dd„ƒZe d%dd„ƒZe d&dd„ƒZe d d!„ƒZe d"d#„ƒZd$S)'Ú DecodeUtilszàUnpacking utility functions used in the decoder. All methods on this class take raw bytes and return a tuple containing the value parsed from the bytes and the number of bytes consumed to parse that value. z!Bz!Hz!Iz!bz!hz!iz!qz!III)ééécCsdS)a This method consumes none of the provided bytes and returns True. :type data: bytes :param data: The bytes to parse from. This is ignored in this method. :rtype: tuple :rtype: (bool, int) :returns: The tuple (True, 0) )Trr ©Údatar r r Ú unpack_trueos zDecodeUtils.unpack_truecCsdS)aThis method consumes none of the provided bytes and returns False. :type data: bytes :param data: The bytes to parse from. This is ignored in this method. :rtype: tuple :rtype: (bool, int) :returns: The tuple (False, 0) )Frr r&r r r Ú unpack_false|s zDecodeUtils.unpack_falsecCs ttj|dd…ƒd}|dfS)zíParse an unsigned 8-bit integer from the bytes. :type data: bytes :param data: The bytes to parse from. :rtype: (int, int) :returns: A tuple containing the (parsed integer value, bytes consumed) Nr#r)rr"ÚUINT8_BYTE_FORMAT©r'Úvaluer r r Ú unpack_uint8‰s zDecodeUtils.unpack_uint8cCs ttj|dd…ƒd}|dfS)zîParse an unsigned 32-bit integer from the bytes. :type data: bytes :param data: The bytes to parse from. :rtype: (int, int) :returns: A tuple containing the (parsed integer value, bytes consumed) Nr%r)rr"ÚUINT32_BYTE_FORMATr+r r r Ú unpack_uint32–s zDecodeUtils.unpack_uint32cCs ttj|dd…ƒd}|dfS)zêParse a signed 8-bit integer from the bytes. :type data: bytes :param data: The bytes to parse from. :rtype: (int, int) :returns: A tuple containing the (parsed integer value, bytes consumed) Nr#r)rr"ÚINT8_BYTE_FORMATr+r r r Ú unpack_int8£s zDecodeUtils.unpack_int8cCs ttj|dd…ƒd}|dfS)aParse a signed 16-bit integer from the bytes. :type data: bytes :param data: The bytes to parse from. :rtype: tuple :rtype: (int, int) :returns: A tuple containing the (parsed integer value, bytes consumed) Nr$r)rr"ÚINT16_BYTE_FORMATr+r r r Ú unpack_int16°s zDecodeUtils.unpack_int16cCs ttj|dd…ƒd}|dfS)aParse a signed 32-bit integer from the bytes. :type data: bytes :param data: The bytes to parse from. :rtype: tuple :rtype: (int, int) :returns: A tuple containing the (parsed integer value, bytes consumed) Nr%r)rr"ÚINT32_BYTE_FORMATr+r r r Ú unpack_int32¾s zDecodeUtils.unpack_int32cCs ttj|dd…ƒd}|dfS)aParse a signed 64-bit integer from the bytes. :type data: bytes :param data: The bytes to parse from. :rtype: tuple :rtype: (int, int) :returns: A tuple containing the (parsed integer value, bytes consumed) Nér)rr"ÚINT64_BYTE_FORMATr+r r r Ú unpack_int64Ìs zDecodeUtils.unpack_int64r$cCs<tj|}t||d|…ƒd}||}|||…}||fS)a¤Parse a variable length byte array from the bytes. The bytes are expected to be in the following format: [ length ][0 ... length bytes] where length is an unsigned integer represented in the smallest number of bytes to hold the maximum length of the array. :type data: bytes :param data: The bytes to parse from. :type length_byte_size: int :param length_byte_size: The byte size of the preceeding integer that represents the length of the array. Supported values are 1, 2, and 4. :rtype: (bytes, int) :returns: A tuple containing the (parsed byte array, bytes consumed). Nr)r"ÚUINT_BYTE_FORMATr)r'Úlength_byte_sizeZuint_byte_formatrZ bytes_endÚ array_bytesr r r Úunpack_byte_arrayÚs   zDecodeUtils.unpack_byte_arraycCst ||¡\}}| d¡|fS)aÙParse a variable length utf-8 string from the bytes. The bytes are expected to be in the following format: [ length ][0 ... length bytes] where length is an unsigned integer represented in the smallest number of bytes to hold the maximum length of the array and the following bytes are a valid utf-8 string. :type data: bytes :param bytes: The bytes to parse from. :type length_byte_size: int :param length_byte_size: The byte size of the preceeding integer that represents the length of the array. Supported values are 1, 2, and 4. :rtype: (str, int) :returns: A tuple containing the (utf-8 string, bytes consumed). zutf-8)r"r<Údecode)r'r:r;Úconsumedr r r Úunpack_utf8_stringós ÿzDecodeUtils.unpack_utf8_stringcCs|dd…dfS)zÛParse a 16-byte uuid from the bytes. :type data: bytes :param data: The bytes to parse from. :rtype: (bytes, int) :returns: A tuple containing the (uuid bytes, bytes consumed). Nér r&r r r Ú unpack_uuid s zDecodeUtils.unpack_uuidcCsttj|ƒtfS)a…Parse the prelude for an event stream message from the bytes. The prelude for an event stream message has the following format: [total_length][header_length][prelude_crc] where each field is an unsigned 32-bit integer. :rtype: ((int, int, int), int) :returns: A tuple of ((total_length, headers_length, prelude_crc), consumed) )rr"ÚPRELUDE_BYTE_FORMATÚ_PRELUDE_LENGTHr&r r r Úunpack_preludes zDecodeUtils.unpack_preludeN)r$)r$)rrr r r*ZUINT16_BYTE_FORMATr.r0r2r4r7rBr9Ú staticmethodr(r)r-r/r1r3r5r8r<r?rArDr r r r r"WsJý           r"cCs$t||ƒd@}||kr t||ƒ‚dS)Nlÿÿ)rr)r'ZchecksumÚcrcZcomputed_checksumr r r Ú_validate_checksum'srGc@s<eZdZdZdd„Zedd„ƒZedd„ƒZedd „ƒZd S) ÚMessagePreludez2Represents the prelude of an event stream message.cCs||_||_||_dS©N)Ú total_lengthÚheaders_lengthrF)rrJrKrFr r r r2szMessagePrelude.__init__cCs|j|jtdS)z§Calculates the total payload length. The extra minus 4 bytes is for the message CRC. :rtype: int :returns: The total payload length. r%)rJrKrC©rr r r Úpayload_length7s zMessagePrelude.payload_lengthcCs |jdS)a Calculates the byte offset for the end of the message payload. The extra minus 4 bytes is for the message CRC. :rtype: int :returns: The byte offset from the beginning of the event stream message to the end of the payload. r%)rJrLr r r Ú payload_endBs zMessagePrelude.payload_endcCs t|jS)zÐCalculates the byte offset for the end of the message headers. :rtype: int :returns: The byte offset from the beginning of the event stream message to the end of the headers. )rCrKrLr r r Ú headers_endNszMessagePrelude.headers_endN) rrr r rÚpropertyrMrNrOr r r r rH/s  rHc@s"eZdZdZdd„Zddd„ZdS) ÚEventStreamMessagez#Represents an event stream message.cCs||_||_||_||_dSrI)ÚpreludeÚheadersÚpayloadrF)rrRrSrTrFr r r r\szEventStreamMessage.__init__éÈcCs0|j d¡}|dks|dkr d}||j|jdœS)Nz :message-typeÚerrorÚ exceptioni)Ú status_coderSÚbody)rSÚgetrT)rrXZ message_typer r r Úto_response_dictbs ýz#EventStreamMessage.to_response_dictN)rU)rrr r rr[r r r r rQYsrQc @s~eZdZdZejejejejej ej ej ej ej ej dœ Zdd„Zdd„Zdd„Zd d „Zd d „Zd d„Zdd„Zdd„ZdS)ÚEventStreamHeaderParseraParses the event headers from an event stream message. Expects all of the header data upfront and creates a dictionary of headers to return. This object can be reused multiple times to parse the headers from multiple event stream messages. ) rr#r$ér%ééér6é cCs d|_dSrI©Ú_datarLr r r rŽsz EventStreamHeaderParser.__init__cCs||_| ¡S)aParses the event stream headers from an event stream message. :type data: bytes :param data: The bytes that correspond to the headers section of an event stream message. :rtype: dict :returns: A dicionary of header key, value pairs. )rcÚ_parse_headers©rr'r r r Úparse‘s zEventStreamHeaderParser.parsecCs4i}|jr0| ¡\}}||kr&t|ƒ‚|||<q|SrI)rcÚ _parse_headerr )rrSÚnamer,r r r rdžs  z&EventStreamHeaderParser._parse_headerscCs| ¡}| ¡}||fSrI)Ú _parse_nameÚ _parse_value)rrhr,r r r rg§sz%EventStreamHeaderParser._parse_headercCs t |jd¡\}}| |¡|S)Nr#)r"r?rcÚ _advance_data)rrhr>r r r ri¬s z#EventStreamHeaderParser._parse_namecCst |j¡\}}| |¡|SrI)r"r-rcrk)rÚtyper>r r r Ú _parse_type±s z#EventStreamHeaderParser._parse_typecCs.| ¡}|j|}||jƒ\}}| |¡|SrI)rmÚ_HEADER_TYPE_MAPrcrk)rZ header_typeZvalue_unpackerr,r>r r r rj¶s   z$EventStreamHeaderParser._parse_valuecCs|j|d…|_dSrIrb)rr>r r r rk½sz%EventStreamHeaderParser._advance_dataN)rrr r r"r(r)r1r3r5r8r<r?rArnrrfrdrgrirmrjrkr r r r r\ms( ì  r\c@s€eZdZdZdd„Zdd„Zdd„Zdd „Zd d „Zd d „Z dd„Z dd„Z dd„Z dd„Z dd„Zdd„Zdd„Zdd„ZdS)ÚEventStreamBufferz´Streaming based event stream buffer A buffer class that wraps bytes from an event stream providing parsed messages as they become available via an iterable interface. cCsd|_d|_tƒ|_dS)Nó)rcÚ_preluder\Ú_header_parserrLr r r rÈszEventStreamBuffer.__init__cCs|j|7_dS)z‡Add data to the buffer. :type data: bytes :param data: The bytes to add to the buffer to be used when parsing Nrbrer r r Úadd_dataÍszEventStreamBuffer.add_datacCs,|jtkrt|jƒ‚|jtkr(t|jƒ‚dSrI)rKrrrMrr)rrRr r r Ú_validate_preludeÕs   z#EventStreamBuffer._validate_preludecCsJ|jdt…}t |¡\}}t|Ž}| |¡t|dtd…|jƒ|S©Nr%)rcrCr"rDrHrtrGrF)rZ prelude_bytesZ raw_preludeÚ_rRr r r Ú_parse_preludeÜs  z EventStreamBuffer._parse_preludecCs|jt|jj…}|j |¡SrI)rcrCrqrOrrrf)rÚ header_bytesr r r rdåsz EventStreamBuffer._parse_headerscCs|j}|j|j|j…}|SrI)rqrcrOrN)rrRZ payload_bytesr r r Ú_parse_payloadész EventStreamBuffer._parse_payloadcCs*|j}|j|j|j…}t |¡\}}|SrI)rqrcrNrJr"r/)rrRZ crc_bytesÚ message_crcrvr r r Ú_parse_message_crcîsz$EventStreamBuffer._parse_message_crccCs|jtd|jj…}|Sru)rcrCrqrN)rÚ message_bytesr r r Ú_parse_message_bytesôs ÿz&EventStreamBuffer._parse_message_bytescCs&| ¡}| ¡}t|||jjd|S)N)rF)r{r}rGrqrF)rrzr|r r r Ú_validate_message_crcûsz'EventStreamBuffer._validate_message_crccCs4| ¡}| ¡}| ¡}t|j|||ƒ}| ¡|SrI)r~rdryrQrqÚ_prepare_for_next_message)rrFrSrTrr r r Ú_parse_messages z EventStreamBuffer._parse_messagecCs|j|jjd…|_d|_dSrI)rcrqrJrLr r r r sz+EventStreamBuffer._prepare_for_next_messagecCsHt|jƒtkrtƒ‚|jdkr(| ¡|_t|jƒ|jjkr@tƒ‚| ¡S)z—Provides the next available message parsed from the stream :rtype: EventStreamMessage :returns: The next event stream message N)ÚlenrcrCÚ StopIterationrqrwrJr€rLr r r Únexts  zEventStreamBuffer.nextcCs| ¡SrI)rƒrLr r r Ú__next__szEventStreamBuffer.__next__cCs|SrIr rLr r r Ú__iter__"szEventStreamBuffer.__iter__N)rrr r rrsrtrwrdryr{r}r~r€rrƒr„r…r r r r roÁs roc@s@eZdZdZdd„Zdd„Zdd„Zdd „Zd d „Zd d „Z dS)Ú EventStreamaÐWrapper class for an event stream body. This wraps the underlying streaming body, parsing it for individual events and yielding them as they come available through the iterator interface. The following example uses the S3 select API to get structured data out of an object stored in S3 using an event stream. **Example:** :: from botocore.session import Session s3 = Session().create_client('s3') response = s3.select_object_content( Bucket='bucketname', Key='keyname', ExpressionType='SQL', RequestProgress={'Enabled': True}, Expression="SELECT * FROM S3Object s", InputSerialization={'CSV': {}}, OutputSerialization={'CSV': {}}, ) # This is the event stream in the response event_stream = response['Payload'] end_event_received = False with open('output', 'wb') as f: # Iterate over events in the event stream as they come for event in event_stream: # If we received a records event, write the data to a file if 'Records' in event: data = event['Records']['Payload'] f.write(data) # If we received a progress event, print the details elif 'Progress' in event: print(event['Progress']['Details']) # End event indicates that the request finished successfully elif 'End' in event: print('Result is complete') end_event_received = True if not end_event_received: raise Exception("End event not received, request incomplete.") cCs&||_||_||_||_| ¡|_dSrI)Ú _raw_streamÚ _output_shapeÚ_operation_nameÚ_parserÚ_create_raw_event_generatorÚ_event_generator)rZ raw_streamZ output_shapeÚparserZoperation_namer r r rRs zEventStream.__init__ccs$|jD]}| |¡}|r|VqdSrI)rŒÚ _parse_event)rÚeventZ parsed_eventr r r r…Ys  zEventStream.__iter__ccs.tƒ}|j ¡D]}| |¡|EdHqdSrI)ror‡Ústreamrs)rZevent_stream_bufferÚchunkr r r r‹_s z'EventStream._create_raw_event_generatorcCs8| ¡}|j ||j¡}|ddkr(|St||jƒ‚dS)NrXrU)r[rŠrfrˆrr‰)rrZ response_dictZparsed_responser r r rŽes  zEventStream._parse_eventcCsHz(t|jƒ}|j d¡}|dkr&|WSWntk r<YnXtƒ‚dS)Nz :event-typezinitial-response)rƒrŒrSrZr‚r!)rZ initial_eventZ event_typer r r Úget_initial_responsems   z EventStream.get_initial_responsecCs|j ¡dS)z%Closes the underlying streaming body.N)r‡ÚcloserLr r r r“wszEventStream.closeN) rrr r rr…r‹rŽr’r“r r r r r†&s+ r†N)r)r ÚbinasciirÚstructrZbotocore.exceptionsrrCrrÚ Exceptionrr rrrr!r"rGrHrQr\ror†r r r r Ú s(       Q *Te