U ,-e@sdZddlZddlZddlZddlmZmZmZmZddl Z ddl Z ddl m Z ddl m Z ddlmZmZmZddlmZdd lmZmZmZdd lmZdd lmZmZmZdd lmZm Z m!Z!m"Z"d dl#m$Z$e!%e&Z'dZ(dZ)dddgZ*ddZ+Gddde j,Z-Gddde j,Z.Gddde j,Z/Gddde j,Z0GdddeZ1dZ2d Z3ed!e2Gd"d#d#e1Z4ed$e2Gd%d&d&e1Z5ed'e2Gd(d)d)e1Z6dS)*zPyTorch OpenAI ImageGPT model.N)AnyOptionalTupleUnion)nn)autocast)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN))BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentions SequenceClassifierOutputWithPast)PreTrainedModel)Conv1D find_pruneable_heads_and_indicesprune_conv1d_layer)add_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings)ImageGPTConfigzopenai/imagegpt-smallrzopenai/imagegpt-mediumzopenai/imagegpt-largec Cszddl}ddl}Wn tk r4tdYnXtj|}td ||j |}g}g}|D]@\} } td | | |j || } | | | | qjt||D]$\} } | dd} | d} tdd | Ds| d d krtd d| q|} | d d kr*t| d} | D]} |d| rP|d| }n| g}|ddksr|ddkrt| d} n|ddkrt| d} n|ddks|ddkrt| |d} t| d} n|ddkrt| d} t| d} nt| dkr8| ddkr8|ddkr8t| |d} t| d} nV|ddkr\t| d} t| d} n2|dd krt| d} t| d} nt| |d} t|d!kr.t|d}| |} q.t| dkr| ddksP| d dksP| d d ksP| d dkrnTz| j| jkstWn<tk rN}z|j| j| jf7_W5d}~XYnXtd" | | d d#krt| |j|jj| jddd|jf<q| d d$krt| |j|jj| jdd|jd!|jf<q| d d%krt| |j|jj| jddd!|jdf<qt| dkrb| ddkrb| d!dkrbt| |j|j| _q| d dkr~t| | _q| d dkrt| | jd|j dddf<q| d d krt| | jd <qt| | _q|S)&z0 Load tf checkpoints in a pytorch model rNzLoading a TensorFlow model in PyTorch, requires TensorFlow to be installed. Please see https://www.tensorflow.org/install/ for installation instructions.z(Converting TensorFlow checkpoint from {}z"Loading TF weight {} with shape {}/css|]}|dkVqdS))Zadam_vZadam_mZAdamWeightDecayOptimizerZAdamWeightDecayOptimizer_1Z global_stepN).0nrro/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/transformers/models/imagegpt/modeling_imagegpt.py Vsz.load_tf_weights_in_imagegpt..)Z_stepz Skipping {})wtet transformerz [A-Za-z]+\d+z(\d+)wgweightbbiaswpewte)q_projk_projv_projc_attnr rattnc_projr"lm_headsoszInitialize PyTorch weight {}r+r,r-)!reZ tensorflow ImportErrorloggererrorospathabspathinfoformattrainZlist_variablesZ load_variableappendsqueezezipsplitanyjoingetattr fullmatchlenintshapeAssertionErrorargstorchZ from_numpyreshapen_embdTdata vocab_size)modelconfigZimagegpt_checkpoint_pathr4tfZtf_pathZ init_varsnamesZarraysnamerHarrayZpointerZm_nameZ scope_namesnumerrrload_tf_weights_in_imagegpt5s              *       F."2*$rYcs<eZdZdeeedfdd Zeje dddZ Z S) ImageGPTLayerNormh㈵>) hidden_sizeepscs&t||_tt||_dSN)super__init__r]r ParameterrKTensorr&)selfr\r] __class__rrr`s zImageGPTLayerNorm.__init__)tensorreturncCs:|ttjt|ddd|j|jjdddfS)Nr!T)ZaxisZkeepdim.)rKsqrtmeanZsquarer]r&rO)rcrfrrrforwards  zImageGPTLayerNorm.forward)r[) __name__ __module__ __qualname__rrGfloatr`rKrbtuplerj __classcell__rrrdrrZsrZc seZdZdeeeedfdd ZddZddd Zdd d Z d d Z ddZ de j eeee j ee j ee j ee j eeeeed ddZZS)ImageGPTAttentionFNis_cross_attention layer_idxcsFt|j}|jdttj||ftjddd||dd|jdt ddd|j |_ |j |_ |j |j |_|j |_|j|j |j krtd|j d |j d |j|_||_|j|_||_|j|_|jrtd |j |j |_t|j |j |_ntd |j |j |_t|j |j |_t|j|_t|j|_t |_!dS) Nr(dtyperF) persistentZ masked_biasgz=`embed_dim` must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).r3r )"r_r`max_position_embeddingsZregister_bufferrKZtrilonesboolviewrfr\ embed_dimZnum_attention_heads num_headshead_dim split_size ValueErrorscale_attn_weightsrsscale_attn_by_inverse_layer_idxrtreorder_and_upcast_attnrr.q_attnr0rDropoutZ attn_pdrop attn_dropout resid_pdrop resid_dropoutset pruned_heads)rcrRrsrtZ max_positionsrdrrr`sD zImageGPTAttention.__init__cCst|dkrdSt||j|j|j\}}t|||j|d|jg}t|j |dd|_ t|j |dd|_ |j|j|jt||_|jt||_|j ||_dS)Nrr3rdim) rFrr}r~rrKcatrrr.r0union)rcheadsindexZ index_attnrrr prune_headss  zImageGPTAttention.prune_headsc Cst||dd}|jr0|t|dd}|jrH|t|jd}|js|d|d}}|j dddd|||d|f} t |j j } tj | |j d|j} t| || }|dk r||}tjdd|}||j }||}|dk r||}t||} | |fS)Nr!?rrur)rKmatmul transposerrnsizerrtrsr(finforvminrftodevicewhererSoftmaxtyper) rcquerykeyvalueattention_mask head_mask attn_weights query_length key_length causal_mask mask_value attn_outputrrr_attns(&    zImageGPTAttention._attnc Cs|\}}}} |\} } } } tj|||| tj|jd} d} |jr\| t|dd} |jrt| t|jd} t ddX| d|| | dd d| | }}tj | ||d | d } | |||| } W5QRX|j sT|d|d}}|jdddd|||d|f}t| jj}tj|| jd | j}t|| |} |dk rf| |} tjdd | } | jtjkrtd | |j} || } |dk r| |} t| |}|| fS)Nrvr?r!rrF)enabledrr)betaalpharurzDError with upcasting, attn_weights does not have dtype torch.float32)rrKemptyZfloat32rrrnrrtrrLrZbaddbmmrsr(rrvrrfrrrr RuntimeErrorrrr)rcrrrrrZbszr}Z q_seq_lenZdk_Z k_seq_lenrZ scale_factorqkrrrrrrrr_upcast_and_reordered_attns: &&     z,ImageGPTAttention._upcast_and_reordered_attncCs2|dd||f}|j|}|ddddS)zJ Splits hidden_size dim into attn_head_size and num_heads Nr!rr3rr )rr{permutercrfr}Zattn_head_sizeZ new_shaperrr _split_headsEs zImageGPTAttention._split_headscCs8|dddd}|dd||f}||S)zS Merges attn_head_size dim and num_attn_heads dim into hidden_size rr3rr Nr)r contiguousrr{rrrr _merge_headsMszImageGPTAttention._merge_heads hidden_states layer_pastrrencoder_hidden_statesencoder_attention_mask use_cacheoutput_attentionsrgc CsV|dk rDt|dstd||} ||j|jdd\} } |}n||j|jdd\} } } || |j|j} || |j|j} || |j|j} |dk r|\} } t j | | fdd} t j | | fdd} |dkr| | f}nd}|j r| | | | ||\}}n| | | | ||\}}|||j|j}||}||}||f}|rR||f7}|S)NrzIf class is used as cross attention, the weights `q_attn` have to be defined. Please make sure to instantiate class with `ImageGPTAttention(..., is_cross_attention=True)`.r3rrT)hasattrrrr.rArrr}r~rKrrrrrr0r)rcrrrrrrrrrrrZpast_keyZ past_valueZpresentrroutputsrrrrjUs:       zImageGPTAttention.forward)FN)NN)NN)NNNNNFF)rkrlrmrrzrGr`rrrrrrKrbrorjrprrrdrrqs0+ & 4 rqcs0eZdZfddZejejdddZZS) ImageGPTMLPcsFt|j}t|||_t|||_t|j|_t |j |_ dSr^) r_r`r\rc_fcr0r Zactivation_functionactrrrdropout)rcZintermediate_sizerRr|rdrrr`s     zImageGPTMLP.__init__)rrgcCs,||}||}||}||}|Sr^)rrr0r)rcrrrrrjs     zImageGPTMLP.forward)rkrlrmr`rKrbrjrprrrdrrs rc sdeZdZdfdd Zd ejeeeejeejeejeejeeeeed ddZ Z S) ImageGPTBlockNcst|j}|jdk r |jnd|}t||jd|_t||d|_t||jd|_ |j r|t|d|d|_ t||jd|_ t |||_dS)Nr]rtTrr)r_r`r\Zn_innerrZlayer_norm_epsilonln_1rqr/ln_2add_cross_attentioncrossattention ln_cross_attnrmlp)rcrRrtr\Z inner_dimrdrrr`s zImageGPTBlock.__init__Frc Cs|} ||}|j||||||d} | d} | dd} | | }|dk rt|dsbtd|d|} ||}|j||||||d} | d} | | }| | dd} |} ||}||}| |}|f|r| n | dd} | S) N)rrrrrrrrz'If `encoder_hidden_states` are passed, z` has to be instantiated with cross-attention layers by setting `config.add_cross_attention=True`)rrrrrr3)rr/rrrrrr)rcrrrrrrrrZresidualZ attn_outputsrrZcross_attn_outputsZfeed_forward_hidden_statesrrrrjsJ        zImageGPTBlock.forward)N)NNNNNFF) rkrlrmr`rKrbrrzrorjrprrrdrrs&rcsFeZdZdZeZeZdZdZ dZ fddZ ddZ d d d Z ZS) ImageGPTPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. r# input_idsTcstj||dSr^)r_r`)rcinputskwargsrdrrr`sz ImageGPTPreTrainedModel.__init__cCst|tjtfr>|jjjd|jjd|j dk r|j j nXt|tj r~|jjjd|jjd|j dk r|jj|j  nt|t r|jjd|D]>\}}d|krd|kr|jjd|jjtd|jjdqdS)zInitialize the weights.g)riZstdNrr0r&r3) isinstancerLinearrr&rOZnormal_rRZinitializer_ranger(Zzero_ EmbeddingZ padding_idxrZZfill_Znamed_parametersmathrhn_layer)rcmodulerUprrr _init_weightss    z%ImageGPTPreTrainedModel._init_weightsFcCst|tr||_dSr^)r ImageGPTModelgradient_checkpointing)rcrrrrr_set_gradient_checkpointings z3ImageGPTPreTrainedModel._set_gradient_checkpointing)F)rkrlrm__doc__r config_classrYZload_tf_weightsZbase_model_prefixZmain_input_nameZsupports_gradient_checkpointingr`rrrprrrdrrs raB This model inherits from [`PreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`ImageGPTConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): `input_ids_length` = `sequence_length` if `past_key_values` is `None` else `past_key_values[0][0].shape[-2]` (`sequence_length` of input past key value states). Indices of input sequence tokens in the vocabulary. If `past_key_values` is used, only `input_ids` that do not have their past calculated should be passed as `input_ids`. Indices can be obtained using [`AutoImageProcessor`]. See [`ImageGPTImageProcessor.__call__`] for details. past_key_values (`Tuple[Tuple[torch.Tensor]]` of length `config.n_layers`): Contains precomputed hidden-states (key and values in the attention blocks) as computed by the model (see `past_key_values` output below). Can be used to speed up sequential decoding. The `input_ids` which have their past given to this model should not be passed as `input_ids` as they have already been computed. attention_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. If `past_key_values` is used, optionally only the last `inputs_embeds` have to be input (see `past_key_values`). use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zbThe bare ImageGPT Model transformer outputting raw hidden-states without any specific head on top.cseZdZedfdd ZddZddZdd Zee e e e d de eje eeeje eje eje eje eje eje eje eje ee ee ee eeeee fd d dZZS)rrRcstj|_tj|j|_tj|j|_ t j |_ t fddtjD|_t|jjd|_d|_d|_d|_|dS)Ncsg|]}t|dqS)r)r)rirrr rsz*ImageGPTModel.__init__..rF)r_r`r\r|rrrPr*rxr)rZ embd_pdropdropZ ModuleListrangeZnum_hidden_layershrZrln_fmodel_parallel device_mapr post_initrcrRrdrrr`is  zImageGPTModel.__init__cCs|jSr^r*rcrrrget_input_embeddings|sz"ImageGPTModel.get_input_embeddingscCs ||_dSr^rrcZnew_embeddingsrrrset_input_embeddingssz"ImageGPTModel.set_input_embeddingscCs(|D]\}}|j|j|qdS)zv Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} N)itemsrr/r)rcZheads_to_prunelayerrrrr _prune_headsszImageGPTModel._prune_heads output_typerN)rpast_key_valuesrtoken_type_ids position_idsr inputs_embedsrrrroutput_hidden_states return_dictrrgc% sDd|kr.tdt|dk r$td|d}dk r:n|jj| dk rN| n|jj} dk rbn|jj| dk rv| n|jj } |dk r|dk rtdnd|dk r| ||| }| d|d}|j d}n,|dk r| dd}|j d}ntd|dk r |jn|j}|dk r,| d|d}|dk rF| d|d}|dkrjd}tdgt|j}n|dd d }|dkrtj||d|tj|d }|d d|d}|dk r|dkrtd | |d}|ddddddf}|j|jd }d |t|jj}|jjrj|dk rj| \}}}||f}| dkr^tj||d} || } nd} |||jj}|dkr||}| |}|||dk r||}||!| df}|j"r|j#rrt$%ddrdnd}rdnd}r.|jjr.dnd}| r>> from transformers import AutoImageProcessor, ImageGPTModel >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("openai/imagegpt-small") >>> model = ImageGPTModel.from_pretrained("openai/imagegpt-small") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_states = outputs.last_hidden_state ``` pixel_valueskThe `pixel_values` argument is deprecated and will be removed in a future version, use `input_ids` instead.N_You cannot pass both `pixel_values` and `input_ids`. Please make sure to only pass `input_ids`.zDYou cannot specify both input_ids and inputs_embeds at the same timer!rz5You have to specify either input_ids or inputs_embedsrrz$batch_size has to be defined and > 0rur)rzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Frc3s|]}|jVqdSr^)rrrZ past_state)rrrr +sz(ImageGPTModel.forward..csfdd}|S)Ncs|fSr^r)r)rrrrrcustom_forward7szLImageGPTModel.forward..create_custom_forward..custom_forwardr)rr)rr)rrcreate_custom_forward6sz4ImageGPTModel.forward..create_custom_forward)rrrrrrrTrr3r zcuda:css|]}|dk r|VqdSr^r)rvrrrr is)Zlast_hidden_staterr attentionscross_attentions)5warningswarn FutureWarningrpoprRrrruse_return_dictZ%warn_if_padding_and_no_attention_maskrr{rHrrorFrrKZarangelong unsqueezerrvrrrryZinvert_attention_maskZ get_head_maskrr*r)rrZtrainingr6Z warning_once enumerater@rcudaZ set_devicerrbutils checkpointrrstrZ last_devicerr )%rcrrrrrrrrrrrrrrZ input_shapeZ batch_sizerZ past_lengthZencoder_batch_sizeZencoder_sequence_lengthrZencoder_hidden_shapeZposition_embedsZtoken_type_embedsZ output_shapeZpresentsZall_self_attentionsZall_cross_attentionsZall_hidden_statesrblockrrrrrr)rrrrrjs-                             "    zImageGPTModel.forward) NNNNNNNNNNNNN)rkrlrmrr`rrrrIMAGEGPT_INPUTS_DOCSTRINGrr _CONFIG_FOR_DOCrrKrbrrzrrrjrprrrdrrdsH  rz The ImageGPT Model transformer with a language modeling head on top (linear layer with weights tied to the input embeddings). cseZdZdgZedfdd ZddZddZdej e e d d d Z e eeeed de ej e eeej e ej e ej e ej e ej e ej e ej e ej e ej e e e e e e e e eeeefdddZeeeej ej eeej dddZZS)ImageGPTForCausalImageModelingzlm_head.weightrcsHt|t||_tj|j|jddd|_d|_ d|_ | dS)NrFr() r_r`rr#rrrMrPr1rrrrrdrrr`s   z'ImageGPTForCausalImageModeling.__init__cCs|jSr^r1rrrrget_output_embeddingssz4ImageGPTForCausalImageModeling.get_output_embeddingscCs ||_dSr^rrrrrset_output_embeddingssz4ImageGPTForCausalImageModeling.set_output_embeddingsN)rrcKs|dd}|rD|dddfd}|dk rD|dddfd}|dd}|dd}|dk r|dkr|dd}||dkd|r|dddfd}nd}|||d|||dS) Nrr!rrrrr)rrrrrr)getr r ZcumsumZ masked_fill_)rcrrrrrrrrrprepare_inputs_for_generations(   z>> from transformers import AutoImageProcessor, ImageGPTForCausalImageModeling >>> import torch >>> import matplotlib.pyplot as plt >>> import numpy as np >>> image_processor = AutoImageProcessor.from_pretrained("openai/imagegpt-small") >>> model = ImageGPTForCausalImageModeling.from_pretrained("openai/imagegpt-small") >>> device = torch.device("cuda" if torch.cuda.is_available() else "cpu") >>> model.to(device) # doctest: +IGNORE_RESULT >>> # unconditional generation of 8 images >>> batch_size = 4 >>> context = torch.full((batch_size, 1), model.config.vocab_size - 1) # initialize with SOS token >>> context = context.to(device) >>> output = model.generate( ... input_ids=context, max_length=model.config.n_positions + 1, temperature=1.0, do_sample=True, top_k=40 ... ) >>> clusters = image_processor.clusters >>> height = image_processor.size["height"] >>> width = image_processor.size["width"] >>> samples = output[:, 1:].cpu().detach().numpy() >>> samples_img = [ ... np.reshape(np.rint(127.5 * (clusters[s] + 1.0)), [height, width, 3]).astype(np.uint8) for s in samples ... ] # convert color cluster tokens back to pixels >>> f, axes = plt.subplots(1, batch_size, dpi=300) >>> for img, ax in zip(samples_img, axes): # doctest: +IGNORE_RESULT ... ax.axis("off") ... ax.imshow(img) ```rrNr) rrrrrrrrrrrrr.r!r)losslogitsrrrr)rrr rr rRr r#r1rr r{rrrrrr)rcrrrrrrrrrrrrrrrtransformer_outputsrZ lm_logitsrZ shift_logitsZ shift_labelsloss_fctoutputrrrrjsZ@  z&ImageGPTForCausalImageModeling.forward)rbeam_idxrgcstfdd|DS)a This function is used to re-order the `past_key_values` cache if [`~PreTrainedModel.beam_search`] or [`~PreTrainedModel.beam_sample`] is called. This is required to match `past_key_values` with the correct beam_idx at every generation step. c3s$|]}tfdd|DVqdS)c3s"|]}|d|jVqdS)rN)Z index_selectrrrr#rrr 2szJImageGPTForCausalImageModeling._reorder_cache...Nro)rrr$rrr 1sz@ImageGPTForCausalImageModeling._reorder_cache..r%)rr#rr$r_reorder_cache(s z-ImageGPTForCausalImageModeling._reorder_cache)N)NNNNNNNNNNNNNN)rkrlrmZ_tied_weights_keysrr`rrrKrbrrzrrrrrrrrrrj staticmethodr&rprrrdrrxsX   w  rz The ImageGPT Model transformer with an image classification head on top (linear layer). [`ImageGPTForImageClassification`] average-pools the hidden states in order to do the classification. cseZdZedfdd Zeeeee dd e e j e e e e j e e j e e j e e j e e j e e j e e j e ee ee ee eeee efdddZZS) ImageGPTForImageClassificationrcs@t||j|_t||_tj|j|jdd|_| dS)NFr) r_r` num_labelsrr#rrrMscorerrrdrrr`?s   z'ImageGPTForImageClassification.__init__rN)rrrrrrrrrrrrrrgc  Ksd| kr.tdt|dk r$td| d}| dk r:| n|jj} |j|||||||| | | | d }|d}|jdd}| |}d}|dk rp|jj dkr|j dkrd |j_ n4|j dkr|j t jks|j t jkrd |j_ nd |j_ |jj d kr"t}|j dkr|||}n |||}nN|jj d krRt}||d |j |d }n|jj d krpt}|||}| s|f|dd}|dk r|f|S|St|||j|j|jd S)a7 labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Returns: Examples: ```python >>> from transformers import AutoImageProcessor, ImageGPTForImageClassification >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("openai/imagegpt-small") >>> model = ImageGPTForImageClassification.from_pretrained("openai/imagegpt-small") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits ```rrNr) rrrrrrrrrrrrrZ regressionZsingle_label_classificationZmulti_label_classificationr!)rrrrr)rrr rr rRr r#rir*Z problem_typer)rvrKr rGr r?r r{rrrrr)rcrrrrrrrrrrrrrr rZpooled_hidden_statesrrr!r"rrrrjHsp,        "    z&ImageGPTForImageClassification.forward) NNNNNNNNNNNN)rkrlrmrr`rrrrrrrKrbrrzrrrjrprrrdrr(7s>   r()7rrr8rtypingrrrrrKZtorch.utils.checkpointrZtorch.cuda.amprZtorch.nnrr r Z activationsr Zmodeling_outputsr rrZmodeling_utilsrZ pytorch_utilsrrrrrrrrZconfiguration_imagegptrZ get_loggerrkr6Z_CHECKPOINT_FOR_DOCrZ&IMAGEGPT_PRETRAINED_MODEL_ARCHIVE_LISTrYModulerZrqrrrZIMAGEGPT_START_DOCSTRINGrrrr(rrrrsb      l\K.?9