U 9%ec@sdZddlmZmZmZmZddlZddlZddlmZddl m Z m Z m Z ddl mZddlmZmZmZmZdd lmZdd lmZmZmZmZmZd d lmZeeZ d Z!dZ"d ddgZ#dZ$dZ%dZ&dd dddddgZ'd7ej(ej)ej*e+dddZ,d8ej-ej)ee+dddZ.Gd d!d!ej/Z0Gd"d#d#ej1Z2Gd$d%d%ej1Z3d&Z4ed'e4Gd(d)d)eZ5d*Z6Gd+d,d,e5Z7ed'e4Gd-d.d.e5Z8Gd/d0d0e5Z9ed1e4Gd2d3d3e5Z:ed4e4Gd5d6d6e5Z;dS)9z PyTorch OPT model.)ListOptionalTupleUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPast)PreTrainedModel)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) OPTConfigzfacebook/opt-350mrizArthurZ/opt-350m-dummy-scg\(\?z 'LABEL_0'zfacebook/opt-125mzfacebook/opt-1.3bzfacebook/opt-2.7bzfacebook/opt-6.7bzfacebook/opt-13bzfacebook/opt-30b)input_ids_shapedtypedevicepast_key_values_lengthcCs|\}}tj||ft|j|d}tj|d|d}|||d|ddkd||}|dkrtj tj ||||d|gdd}|ddddddf |d|||S)zB Make causal mask used for bi-directional self-attention. rrr)rrdimN) torchfullfinfominarangesizeZ masked_fill_viewtocatZzerosexpand)rrrrbsztgt_lenmaskZ mask_condr.c/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/transformers/models/opt/modeling_opt.py_make_causal_maskCs"  r0)r-rr,cCsj|\}}|dk r|n|}|ddddddf|d|||}d|}||tjt|jS)z_ Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Nrg?)r&r*r(Z masked_fillr!boolr#r$)r-rr,r+src_lenZ expanded_maskZ inverted_maskr.r.r/ _expand_maskTs  *r3cs@eZdZdZeedfdd Zd ejedfdd ZZ S) OPTLearnedPositionalEmbeddingzN This module learns positional embeddings up to a fixed maximum size. )num_embeddings embedding_dimcsd|_t||j|dSN)offsetsuper__init__)selfr5r6 __class__r.r/r;gsz&OPTLearnedPositionalEmbedding.__init__r)attention_maskrcsN|}tj|dd||d}|dd|df}t||jS)z3`input_ids_shape` is expected to be [bsz x seqlen].rrN)longr!ZcumsumZtype_asr:forwardr9)r<r?rZ positionsr=r.r/rAms z%OPTLearnedPositionalEmbedding.forward)r) __name__ __module__ __qualname____doc__intr;r! LongTensorrA __classcell__r.r.r=r/r4bsr4c seZdZdZdeeeeedfdd Zej eedd d Z dej e ej e e ej e ej e ej ee ej e ej e e ej fd d dZ ZS) OPTAttentionz=Multi-headed attention from 'Attention Is All You Need' paperFT embed_dim num_headsdropout is_decoderbiascst||_||_||_|||_|j||jkrNtd|jd|d|jd|_||_t j |||d|_ t j |||d|_ t j |||d|_ t j |||d|_dS)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩rP)r:r;rLrMrNhead_dim ValueErrorscalingrOrLineark_projv_projq_projout_proj)r<rLrMrNrOrPr=r.r/r;}s   zOPTAttention.__init__)tensorseq_lenr+cCs ||||j|jddS)Nrr8)r'rMrR transpose contiguous)r<rZr[r+r.r.r/_shapeszOPTAttention._shapeN) hidden_stateskey_value_statespast_key_valuer?layer_head_maskoutput_attentionsreturncCs|dk }|\}} } |||j} |rD|dk rD|d} |d} n|rr|||d|} |||d|} n|dk r|||d|} |||d|} tj|d| gdd} tj|d| gdd} n(|||d|} |||d|} |jr| | f}||j d|j f}|| | |j |} | j |} | j |} | d}t | | dd}|||j | |fkrtd||j | |fd||dk r*||d| |fkrtd |d| |fd|| ||j | ||}t|tjt|jj|jd }| ||j | |}|jtjkrVtjj|dtjd tj}ntjj|dd}|dk r||j fkrtd |j fd|| dddd| ||j | |}| ||j | |}|r| ||j | |}| ||j | |}nd}tjj||j|jd }t || }|||j | |j fkrhtd||j | |j fd|| ||j | |j }| dd}||| |j}||}|||fS)z#Input shape: Batch x Time x ChannelNrrrr8rz$Attention weights should be of size z , but is z!Attention mask should be of size r)r rz/Head mask for a single layer should be of size ptrainingz `attn_output` should be of size ) r&rXrTr^rVrWr!r)rOrMrRr'Zbmmr\rSmaxrZr#rr$rZfloat16r functionalZsoftmaxZfloat32r(rNrgreshaperLrY)r<r_r`rar?rbrcZis_cross_attentionr+r,_Z query_statesZ key_statesZ value_statesZ proj_shaper2Z attn_weightsZattn_weights_reshapedZ attn_probsZ attn_outputr.r.r/rAs        "   zOPTAttention.forward)rJFT)NNNNF)rBrCrDrErFfloatr1r;r!Tensorr^rrrArHr.r.r=r/rIzs4 rIc sxeZdZedfdd Zd ejeejeejeeejee ee eej eeej ej ffdddZ Z S) OPTDecoderLayerconfigcst|j|_t|j|j|jd|jd|_|j |_ |j |_ t |j |_ tj|j|jd|_tj|j|j|jd|_tj|j|j|jd|_tj|j|jd|_dS)NTrKZelementwise_affinerQ)r:r; hidden_sizerLrIZnum_attention_headsZattention_dropoutZ enable_bias self_attndo_layer_norm_beforerNr Zactivation_function activation_fnr LayerNormlayer_norm_elementwise_affineself_attn_layer_normrUZffn_dimfc1fc2final_layer_normr<rpr=r.r/r;s&  zOPTDecoderLayer.__init__NF)r_r?rbrarc use_cacherdc Cs|}|jr||}|j|||||d\}}} tjj||j|jd}||}|js\||}|j} |d| d}|}|jr| |}| |}| |}| |}tjj||j|jd}||| }|js| |}|f} |r| |f7} |r| | f7} | S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`, *optional*): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_value (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states )r_rar?rbrcrer)rtrxrsrrirNrgshaperjr&r{ryrurzr') r<r_r?rbrarcr}ZresidualZself_attn_weightsZpresent_key_valueZhidden_states_shapeoutputsr.r.r/rA*s@          zOPTDecoderLayer.forward)NNNFF) rBrCrDrr;r!rmrrr1 FloatTensorrArHr.r.r=r/rns rnaH This model inherits from [`PreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`OPTConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. zQThe bare OPT Model outputting raw hidden-states without any specific head on top.c@s0eZdZeZdZdZdgZddZd ddZ d S) OPTPreTrainedModelmodelTrncCs||jj}t|tjr>|jjjd|d|jdk rx|jj n:t|tj rx|jjjd|d|j dk rx|jj|j  dS)NrJ)Zmeanstd) rpZinit_std isinstancerrUweightdataZnormal_rPZzero_ Embedding padding_idx)r<modulerr.r.r/ _init_weightss    z OPTPreTrainedModel._init_weightsFcCst|tr||_dSN)r OPTDecodergradient_checkpointing)r<rvaluer.r.r/_set_gradient_checkpointings z.OPTPreTrainedModel._set_gradient_checkpointingN)F) rBrCrDr config_classZbase_model_prefixZsupports_gradient_checkpointingZ_no_split_modulesrrr.r.r.r/rs  ra Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). If you want to change padding behavior, you should read [`modeling_opt._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules in the encoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. c seZdZdZedfdd ZddZddZd d Zde j e e j e e j e e e je e je ee ee ee eeeefd d dZZS)rz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`OPTDecoderLayer`] Args: config: OPTConfig rocstj|_j|_j|_j|_j|_t jj |j|_ t jj|_j jkr~t jjj dd|_nd|_j jkrt jj jdd|_nd|_jrЈjst jjjd|_nd|_t fddtjD|_d|_|dS)NFrQrqcsg|] }tqSr.)rn).0rkror.r/ sz'OPTDecoder.__init__..)r:r;rN layerdrop pad_token_idrZmax_position_embeddingsZmax_target_positions vocab_sizerrword_embed_proj_dim embed_tokensr4rrembed_positionsrU project_out project_inrtZ_remove_final_layer_normrvrwr{Z ModuleListrangeZnum_hidden_layerslayersr post_initr|r=ror/r;s.      zOPTDecoder.__init__cCs|jSrrr<r.r.r/get_input_embeddingsszOPTDecoder.get_input_embeddingscCs ||_dSrrr<rr.r.r/set_input_embeddingsszOPTDecoder.set_input_embeddingscCs`d}|ddkr$t||j|j|d}|dk r\t||j|dd|j}|dkrT|n||}|S)Nrr)rr)r,)r0rrr3r()r<r? input_shape inputs_embedsrZcombined_attention_maskZexpanded_attn_maskr.r.r/_prepare_decoder_attention_masks z*OPTDecoder._prepare_decoder_attention_maskN input_idsr? head_maskpast_key_valuesrr}rcoutput_hidden_states return_dictrdc  sdk r n|jj|dk r |n|jj}|dk r4|n|jj}| dk rH| n|jj} |dk rj|dk rjtdnD|dk r|} |d| d}n"|dk r|dd} ntd|dkr||}| \} } |dk r|ddj dnd} | | }|dkr t j | ||j d}n,|j d|kr8td |j dd |d | || || }||| }|jdk rj||}||}|jr|jr|rtd d }|rdnd}rdnd}|rdnd}t|gdgD]V\}}|dk r|dt|jkrtd|dt|jd|ddqt|jD]\}}|rH||f7}|jrjt g}||jkrjq0|dk r|||nd}|jr|jrʇfdd}t jj|||||dk r||ndd}n&||||dk r||nd||d}|d}|r||r dndf7}r0||df7}q0|jdk rD||}|jdk rZ||}|rj||f7}|rt|nd}| stdd||||fDSt||||dS)a" Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(num_hidden_layers, num_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzTYou cannot specify both decoder_input_ids and decoder_inputs_embeds at the same timerzEYou have to specify either decoder_input_ids or decoder_inputs_embedsrr8rrz'The provided attention mask has length z, but its length should be z0 (sum of the lengths of current and past inputs)zZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Fr.rzThe `z` should be specified for z layers, but it is for .csfdd}|S)Ncs|dfSrr.)inputs)rrcr.r/custom_forwardszIOPTDecoder.forward..create_custom_forward..custom_forwardr.)rrrc)rr/create_custom_forwardsz1OPTDecoder.forward..create_custom_forward)r?rbrarcr}css|]}|dk r|VqdSrr.)rvr.r.r/ sz%OPTDecoder.forward..last_hidden_staterr_ attentions) rprcrr}use_return_dictrSr&r'rr~r!ZonesrrrrrrgloggerZ warning_onceziplenr enumerateZrandrutils checkpointr{rtupler )r<rr?rrrr}rcrrr batch_sizeZ seq_lengthrZmask_seq_lengthZcausal_attention_maskZ pos_embedsr_Zall_hidden_statesZall_self_attnsZnext_decoder_cacheZ attn_maskZ mask_nameidxZ decoder_layerZdropout_probabilityrarZ layer_outputsZ next_cacher.rr/rA,s;       $          zOPTDecoder.forward) NNNNNNNNN)rBrCrDrErr;rrrr!rGrrmrrr1rrr rArHr.r.r=r/rs4%  rcseZdZedfdd ZddZddZdd Zee e e e e ed dejeejeejeeejeejeeeeeeeeeee fd d dZZS)OPTModelrocs"t|t||_|dSr)r:r;rdecoderrr|r=r.r/r;s  zOPTModel.__init__cCs|jjSrrrrr.r.r/rszOPTModel.get_input_embeddingscCs ||j_dSrrrr.r.r/rszOPTModel.set_input_embeddingscCs|jSr)rrr.r.r/ get_decoderszOPTModel.get_decoder)r output_typerexpected_outputNrc Cs|dk r |n|jj}|dk r |n|jj}|dk r4|n|jj}| dk rH| n|jj} |j||||||||| d } | st| St| j| j| j | j dS)N rr?rrrr}rcrrr) rprcrr}rrr rrr_r) r<rr?rrrr}rcrrZdecoder_outputsr.r.r/rAs0 zOPTModel.forward) NNNNNNNNN)rBrCrDrr;rrrrOPT_INPUTS_DOCSTRINGr_CHECKPOINT_FOR_DOCr _CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPEr!rGrrmrrr1rrrArHr.r.r=r/rs@  rcseZdZdgZfddZddZddZdd Zd d Zd d Z ddZ e e e ddejeejeejeeejeejeejeeeeeeeeeee fd ddZdddZeddZZS)OPTForCausalLMzlm_head.weightcs8t|t||_tj|j|jdd|_| dSNFrQ) r:r;rrrrUrrlm_headrr|r=r.r/r;5s  zOPTForCausalLM.__init__cCs |jjjSrrrrrr.r.r/r?sz#OPTForCausalLM.get_input_embeddingscCs||jj_dSrrrr.r.r/rBsz#OPTForCausalLM.set_input_embeddingscCs|jSrrrr.r.r/get_output_embeddingsEsz$OPTForCausalLM.get_output_embeddingscCs ||_dSrr)r<Znew_embeddingsr.r.r/set_output_embeddingsHsz$OPTForCausalLM.set_output_embeddingscCs ||j_dSrrr)r<rr.r.r/ set_decoderKszOPTForCausalLM.set_decodercCs|jjSrrrr.r.r/rNszOPTForCausalLM.get_decoderrrN rr?rrrlabelsr}rcrrrdc  Cs|dk r |n|jj}| dk r | n|jj} | dk r4| n|jj} |jj|||||||| | d } || d} d} |dk r|| j }| dddddf}|dddf}t }|| d|jj | d} | s| f| dd}| dk r| f|S|St | | | j| j| jdS)aU Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(num_hidden_layers, num_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. The two additional tensors are only required when the model is used as a decoder in a Sequence to Sequence model. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. Returns: Example: ```python >>> from transformers import AutoTokenizer, OPTForCausalLM >>> model = OPTForCausalLM.from_pretrained("facebook/opt-350m") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious. I'm just a little bit of a weirdo." ```Nrr.rrlosslogitsrr_r)rprcrrrrrr]r(rrr'rr rr_r)r<rr?rrrrr}rcrrrrrZ shift_logitsZ shift_labelsloss_fctoutputr.r.r/rAQsBX  zOPTForCausalLM.forwardcKsV|r|ddddf}|dk r2|dkr2d|i}nd|i}|||d|d|S)Nrrrr})rr}r?)updateget)r<rrr?rkwargsZ model_inputsr.r.r/prepare_inputs_for_generations z,OPTForCausalLM.prepare_inputs_for_generationcs.d}|D] }|tfdd|Df7}q|S)Nr.c3s"|]}|d|jVqdS)rN)Z index_selectr(r)rZ past_statebeam_idxr.r/rsz0OPTForCausalLM._reorder_cache..)r)rrZreordered_pastZ layer_pastr.rr/_reorder_caches zOPTForCausalLM._reorder_cache) NNNNNNNNNN)NNN)rBrCrDZ_tied_weights_keysr;rrrrrrrr rr!rGrrmrrr1rrrAr staticmethodrrHr.r.r=r/r2sN     ra The OPT Model transformer with a sequence classification head on top (linear layer). [`OPTForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). cseZdZedfdd Zeeeee e e e dd e eje eje eje eeeje eje eje ee ee ee eeee fd ddZd d Zd d ZZS)OPTForSequenceClassificationrocs@t||j|_t||_tj|j|jdd|_| dSr) r:r; num_labelsrrrrUrscorerr|r=r.r/r;s   z%OPTForSequenceClassification.__init__)rrrrZ expected_lossNrc  Cs$| dk r | n|jj} |j|||||||| | d } | d} || } |dk r^|jdd\}}n|jdd\}}|jjdkrd}nH|dk rt||jj dd | j }nd}t |jjd| tj|| j d|f}d}|dk r|jjdkrN|jdkrd |j_n:|jdkrF|jtjks<|jtjkrFd |j_nd |j_|jjd krt}|jdkr|||}n |||}nN|jjd krt}||d|j|d}n|jjd krt}|||}| s |f| dd}|dk r|f|S|St||| j| j| jd S) a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrr?rrr}rcrrrr8rrz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`rZ regressionZsingle_label_classificationZmulti_label_classificationr)rprrrr~rr!eqr@Zargmaxr(rrwarningr>rBr%Z problem_typerrrFr squeezerr'rrrr_r)r<rr?rrrrr}rcrrtransformer_outputsr_rrZsequence_lengthZsequence_lengthsZ pooled_logitsrrrr.r.r/rA sr       (    z$OPTForSequenceClassification.forwardcCs |jjjSrrrr.r.r/rosz1OPTForSequenceClassification.get_input_embeddingscCs||jj_dSrrrr.r.r/rrsz1OPTForSequenceClassification.set_input_embeddings) NNNNNNNNNN)rBrCrDrr;rrr'_CHECKPOINT_FOR_SEQUENCE_CLASSIFICATIONrr_SEQ_CLASS_EXPECTED_OUTPUT_SEQ_CLASS_EXPECTED_LOSSrr!rGrrrmr1rrArrrHr.r.r=r/rsD   Zrz The OPT Model transformer with a span classification head on top for extractive question-answering tasks like SQuAD (a linear layers on top of the hidden-states output to compute `span start logits` and `span end logits`). cseZdZedfdd Zeeeee dd e e j e e j e e j e eee je e j e e j e e j e ee ee ee eeeefd ddZd d Zd d ZZS)OPTForQuestionAnsweringrocs2t|t||_t|jd|_|dSr7) r:r;rrrrUr qa_outputsrr|r=r.r/r;~s  z OPTForQuestionAnswering.__init__rN) rr?rrrstart_positions end_positionsr}rcrrrdc  CsP| dk r | n|jj} |j||||||| | | d } | d} || }|jddd\}}|d}|d}d}|dk r|dk rt|dkr|d}t|dkr|d}|d}| d|}| d|}t |d}|||}|||}||d}| s:||f| dd}|dk r6|f|S|St |||| j | j d S) a start_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the start of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. end_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the end of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. Returns: Example: ```python >>> from transformers import AutoTokenizer, OPTForQuestionAnswering >>> import torch >>> torch.manual_seed(4) # doctest: +IGNORE_RESULT >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> # note: we are loading a OPTForQuestionAnswering from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random >>> model = OPTForQuestionAnswering.from_pretrained("facebook/opt-350m") >>> question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" >>> inputs = tokenizer(question, text, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> answer_offset = len(tokenizer(question)[0]) >>> predict_answer_tokens = inputs.input_ids[ ... 0, answer_offset + answer_start_index : answer_offset + answer_end_index + 1 ... ] >>> predicted = tokenizer.decode(predict_answer_tokens) >>> predicted ' a nice puppet' ```Nrrrrr)Z ignore_indexr8)r start_logits end_logitsr_r)rprrrsplitrr]rr&clamprrr_r)r<rr?rrrrrr}rcrrrr_rrrZ total_lossZ ignored_indexrZ start_lossZend_lossrr.r.r/rAsP;           zOPTForQuestionAnswering.forwardcCs |jjjSrrrr.r.r/rsz,OPTForQuestionAnswering.get_input_embeddingscCs||jj_dSrrrr.r.r/rsz,OPTForQuestionAnswering.set_input_embeddings) NNNNNNNNNNN)rBrCrDrr;rrrrrrr!rGrrrmr1rrArrrHr.r.r=r/rvs<  jr)r)N)s       d> BC s