U ,-e}D@sdZddlmZddlZddlZddlmZmZddlmZm Z m Z ddl m Z ddl mZmZmZdd lmZmZmZdd lmZdd lmZd d lmZeeZdZdZd dddgZ dZ!dZ"dgZ#Gdddej$Z%Gdddej$Z&Gdddej$Z'Gdddej$Z(Gdddej$Z)Gdddej$Z*Gdd d ej$Z+Gd!d"d"ej$Z,Gd#d$d$eZ-d%Z.d&Z/ed'e.Gd(d)d)e-Z0ed*e.Gd+d,d,e-Z1dS)-z PyTorch RegNet model.)OptionalN)Tensornn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forward)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)logging) RegNetConfigrzfacebook/regnet-y-040i@ztabby, tabby catcs:eZdZd eeeeeeedfdd ZddZZS) RegNetConvLayerrrrelu) in_channels out_channels kernel_sizestridegroups activationc sRttj|||||d|dd|_t||_|dk rDt|nt|_ dS)NF)rrpaddingrbias) super__init__rConv2d convolution BatchNorm2d normalizationr Identityr)selfrrrrrr __class__k/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/transformers/models/regnet/modeling_regnet.pyr!8s  zRegNetConvLayer.__init__cCs"||}||}||}|SN)r#r%rr' hidden_stater*r*r+forwardNs   zRegNetConvLayer.forward)rrrr) __name__ __module__ __qualname__intrstrr!r/ __classcell__r*r*r(r+r7srcs.eZdZdZedfdd ZddZZS)RegNetEmbeddingszP RegNet Embedddings (stem) composed of a single aggressive convolution. configcs0tt|j|jdd|jd|_|j|_dS)Nrr)rrr)r r!r num_channelsembedding_size hidden_actembedderr'r8r(r*r+r!Zs zRegNetEmbeddings.__init__cCs*|jd}||jkrtd||}|S)NrzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.)shaper9 ValueErrorr<)r' pixel_valuesr9r.r*r*r+r/as   zRegNetEmbeddings.forward)r0r1r2__doc__rr!r/r5r*r*r(r+r6Usr6cs<eZdZdZd eeedfdd ZeedddZZS) RegNetShortCutz RegNet shortcut, used to project the residual features to the correct size. If needed, it is also used to downsample the input using `stride=2`. r)rrrcs0ttj||d|dd|_t||_dS)NrF)rrr)r r!rr"r#r$r%)r'rrrr(r*r+r!rs zRegNetShortCut.__init__)inputreturncCs||}||}|Sr,)r#r%)r'rCr.r*r*r+r/ws  zRegNetShortCut.forward)r) r0r1r2rAr3r!rr/r5r*r*r(r+rBlsrBcs0eZdZdZeedfdd ZddZZS) RegNetSELayerz| Squeeze and Excitation layer (SE) proposed in [Squeeze-and-Excitation Networks](https://arxiv.org/abs/1709.01507). )rreduced_channelsc sLttd|_ttj||ddttj||ddt|_ dS)Nrrr)r) r r!rAdaptiveAvgPool2dpooler Sequentialr"ZReLUZSigmoid attention)r'rrFr(r*r+r!s  zRegNetSELayer.__init__cCs ||}||}||}|Sr,)rIrK)r'r.ZpooledrKr*r*r+r/s  zRegNetSELayer.forward)r0r1r2rAr3r!r/r5r*r*r(r+rE}s rEcs6eZdZdZdeeeedfdd ZddZZS) RegNetXLayerzt RegNet's layer composed by three `3x3` convolutions, same as a ResNet bottleneck layer with reduction = 1. rr8rrrc st||kp|dk}td||j}|rsz(RegNetStage.__init__..r) r r!Z layer_typerLrYrrJrangelayers)r'r8rrrr]r(rar+r!s zRegNetStage.__init__cCs||}|Sr,)rdr-r*r*r+r/s zRegNetStage.forward)rrrXr*r*r(r+r\s r\cs8eZdZedfdd Zd eeeedddZZ S) RegNetEncoderr7c sttg|_|jt||j|jd|j r6dnd|j ddt |j|jdd}t ||j ddD]$\\}}}|jt||||dqrdS)Nrrr)rr])r]) r r!rZ ModuleListstagesappendr\r: hidden_sizesZdownsample_in_first_stageZdepthszip)r'r8Zin_out_channelsrrr]r(r*r+r!s     zRegNetEncoder.__init__FT)r.output_hidden_states return_dictrDcCsb|rdnd}|jD]}|r$||f}||}q|r<||f}|sVtdd||fDSt||dS)Nr*css|]}|dk r|VqdSr,r*)r_vr*r*r+ sz(RegNetEncoder.forward..)last_hidden_state hidden_states)rftupler )r'r.rjrkroZ stage_moduler*r*r+r/s     zRegNetEncoder.forward)FT) r0r1r2rr!rboolr r/r5r*r*r(r+resrec@s2eZdZdZeZdZdZdZddZ d dd Z d S) RegNetPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. regnetr@TcCsXt|tjr"tjj|jdddn2t|tjtjfrTtj|jdtj|j ddS)NZfan_outr)modeZ nonlinearityrr) isinstancerr"initZkaiming_normal_weightr$Z GroupNormZ constant_r)r'moduler*r*r+ _init_weights!s  z#RegNetPreTrainedModel._init_weightsFcCst|tr||_dSr,)ru RegNetModelZgradient_checkpointing)r'rxvaluer*r*r+_set_gradient_checkpointing(s z1RegNetPreTrainedModel._set_gradient_checkpointingN)F) r0r1r2rAr config_classZbase_model_prefixZmain_input_nameZsupports_gradient_checkpointingryr|r*r*r*r+rrsrraH This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`RegNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aK Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`AutoImageProcessor`]. See [`ConvNextImageProcessor.__call__`] for details. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~file_utils.ModelOutput`] instead of a plain tuple. zOThe bare RegNet model outputting raw features without any specific head on top.csTeZdZfddZeeeeee de dd e e e e e edddZZS) rzcs>t|||_t||_t||_td|_ | dS)NrG) r r!r8r6r<reencoderrrHrI post_initr=r(r*r+r!Ls     zRegNetModel.__init__Zvision) checkpoint output_typer}Zmodalityexpected_outputN)r@rjrkrDcCs||dk r |n|jj}|dk r |n|jj}||}|j|||d}|d}||}|sl||f|ddSt|||jdS)Nrjrkrr)rn pooler_outputro)r8rjuse_return_dictr<r~rIrro)r'r@rjrkZembedding_outputZencoder_outputsrn pooled_outputr*r*r+r/Us$   zRegNetModel.forward)NN)r0r1r2r!r REGNET_INPUTS_DOCSTRINGr _CHECKPOINT_FOR_DOCr_CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPErrrqr/r5r*r*r(r+rzFs" rzz RegNet Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. c s`eZdZfddZeeeeee e dde e j e e je ee eedddZZS) RegNetForImageClassificationcs^t||j|_t||_tt|jdkrFt|j d|jnt |_ | dS)Nr) r r! num_labelsrzrsrrJZFlattenZLinearrhr& classifierrr=r(r*r+r!s  $z%RegNetForImageClassification.__init__)rrr}rN)r@labelsrjrkrDc Csl|dk r |n|jj}|j|||d}|r.|jn|d}||}d}|dk r,|jjdkr|jdkrnd|j_n4|jdkr|jtj ks|jtj krd|j_nd|j_|jjdkrt } |jdkr| | | }n | ||}nN|jjdkrt } | |d|j|d}n|jjdkr,t} | ||}|s\|f|dd} |dk rX|f| S| St|||jd S) a0 labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NrrZ regressionZsingle_label_classificationZmulti_label_classificationrr)losslogitsro)r8rrsrrZ problem_typerZdtypetorchlongr3rZsqueezerviewrrro) r'r@rrjrkoutputsrrrZloss_fctoutputr*r*r+r/s6     "     z$RegNetForImageClassification.forward)NNNN)r0r1r2r!r rr _IMAGE_CLASS_CHECKPOINTrr_IMAGE_CLASS_EXPECTED_OUTPUTrrZ FloatTensorZ LongTensorrqr/r5r*r*r(r+rys& r)2rAtypingrrZtorch.utils.checkpointrrZtorch.nnrrrZ activationsr Z file_utilsr r r Zmodeling_outputsr rrZmodeling_utilsrutilsrZconfiguration_regnetrZ get_loggerr0loggerrrrrrZ$REGNET_PRETRAINED_MODEL_ARCHIVE_LISTModulerr6rBrErLrYr\rerrZREGNET_START_DOCSTRINGrrzrr*r*r*r+sP       !& .