U ,-e93@s*dZddlmZddlmZmZddlZddlmZddlm Z ddl m Z dd l m Z mZmZmZdd lmZd d lmZd gZdZeGddde ZGddde ZGdddejZGdddejZGdddejZGdddejZGdddejZdZdZ edeGd d!d!eZ!dS)"z PyTorch ViTMatte model.) dataclass)OptionalTupleN)nn) AutoBackbone)PreTrainedModel) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardreplace_return_docstrings) BackboneMixin)VitMatteConfigz$hustvl/vitmatte-small-composition-1krc@s^eZdZUdZdZeejed<dZ ejed<dZ ee ejed<dZ ee ejed<dS)ImageMattingOutputa Class for outputs of image matting models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Loss. alphas (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Estimated alpha values. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states (also called feature maps) of the model at the output of each stage. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, patch_size, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Nlossalphas hidden_states attentions) __name__ __module__ __qualname____doc__rrtorchZ FloatTensor__annotations__rrrrrro/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/transformers/models/vitmatte/modeling_vitmatte.pyr-s rc@s.eZdZdZeZdZdZddZd ddZ d S) VitMattePreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. pixel_valuesTcCs<t|tjr8|jjjd|jjd|jdk r8|jj dS)Ng)ZmeanZstd) isinstancerConv2dweightdataZnormal_configZinitializer_rangebiasZzero_)selfmodulerrr _init_weightsSs  z%VitMattePreTrainedModel._init_weightsFcCst|tr||_dSN)rr Zgradient_checkpointing)r%r&valuerrr_set_gradient_checkpointingYs z3VitMattePreTrainedModel._set_gradient_checkpointingN)F) rrrrr config_classZmain_input_nameZsupports_gradient_checkpointingr'r*rrrrrIs rcs*eZdZdZdfdd ZddZZS) VitMatteBasicConv3x3zP Basic convolution layers including: Conv3x3, BatchNorm2d, ReLU layers. rcsBttj||d||dd|_tj||jd|_t|_ dS)NrF) in_channels out_channels kernel_sizestridepaddingr$)Zeps) super__init__rr conv BatchNorm2dZbatch_norm_eps batch_normReLUrelu)r%r#r.r/r1r2 __class__rrr4cs zVitMatteBasicConv3x3.__init__cCs"||}||}||}|Sr()r5r7r9r%Z hidden_staterrrforwardps   zVitMatteBasicConv3x3.forward)r-rrrrrr4r= __classcell__rrr:rr,^s r,cs(eZdZdZfddZddZZS)VitMatteConvStreamzc Simple ConvStream containing a series of basic conv3x3 layers to extract detail features. csvt|jj}|j}t|_|g||_t t |jdD]0}|j|}|j|d}|j t |||q@dS)Nr) r3r4backbone_configZ num_channelsconvstream_hidden_sizesr ModuleListconvs conv_chansrangelenappendr,)r%r#r.r/iZin_chan_Z out_chan_r:rrr4}s    zVitMatteConvStream.__init__cCsJd|i}|}tt|jD]*}|j||}dt|d}|||<q|S)NZdetailed_feature_map_0detailed_feature_map_r)rFrGrDstr)r%rZout_dictZ embeddingsrIZname_rrrr=s zVitMatteConvStream.forwardr>rrr:rr@xs r@cs(eZdZdZfddZddZZS)VitMatteFusionBlockz\ Simple fusion block to fuse features from ConvStream and Plain Vision Transformer. cs"tt|||ddd|_dS)Nr)r1r2)r3r4r,r5)r%r#r.r/r:rrr4s zVitMatteFusionBlock.__init__cCs4tjj|dddd}tj||gdd}||}|S)Nr-ZbilinearF)Z scale_factormodeZ align_cornersr)dim)rZ functionalZ interpolatercatr5)r%featuresZdetailed_feature_mapZupscaled_featuresoutrrrr=s zVitMatteFusionBlock.forwardr>rrr:rrLs rLcs(eZdZdZfddZddZZS) VitMatteHeadzJ Simple Matting Head, containing only conv3x3 and conv1x1 layers. c sZt|jd}d}ttj||ddddt|tdtj|ddddd|_dS)Nrr)r0r1r2Tr) r3r4fusion_hidden_sizesrZ Sequentialr r6r8 matting_convs)r%r#r.Z mid_channelsr:rrr4s  zVitMatteHead.__init__cCs||}|Sr()rVr<rrrr=s zVitMatteHead.forwardr>rrr:rrRs rRcs(eZdZdZfddZddZZS)VitMatteDetailCaptureModulezG Simple and lightweight Detail Capture Module for ViT Matting. c stt|jt|jdkr*td||_t||_|jj |_ t |_ |j g|j|_tt|jdD]:}|j t||j||j |d |j|ddqpt||_dS)Nrz_The length of fusion_hidden_sizes should be equal to the length of convstream_hidden_sizes + 1.)r#r.r/)r3r4rGrUrB ValueErrorr#r@ convstreamrErrC fusion_blocksZ hidden_sizeZfusion_channelsrFrHrLrR matting_head)r%r#rIr:rrr4s&     z$VitMatteDetailCaptureModule.__init__cCs`||}tt|jD]2}dtt|j|d}|j||||}qt||}|S)NrJr)rYrFrGrZrKrZsigmoidr[)r%rPrZdetail_featuresrIZdetailed_feature_map_namerrrrr=s  z#VitMatteDetailCaptureModule.forwardr>rrr:rrWs rWaI Parameters: This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) sub-class. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. config ([`UperNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aw Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Padding will be ignored by default should you provide it. Pixel values can be obtained using [`AutoImageProcessor`]. See [`VitMatteImageProcessor.__call__`] for details. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers in case the backbone has them. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers of the backbone. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zNViTMatte framework leveraging any vision backbone e.g. for ADE20k, CityScapes.c sfeZdZfddZeedeee dd e e j e e e e e e j e e dddZZS) VitMatteForImageMattingcs6t|||_t|j|_t||_| dSr() r3r4r#r from_configrAbackbonerWdecoder post_init)r%r#r:rrr4s   z VitMatteForImageMatting.__init__zbatch_size, sequence_length) output_typer+N)routput_attentionsoutput_hidden_stateslabels return_dictc Cs|dk r |n|jj}|dk r |n|jj}|dk r4|n|jj}|jj|||d}|jd}|||}d} |dk rxtd|s|f|dd} | dk r| f| S| St | ||j |j dS)aJ labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth image matting for computing the loss. Returns: Examples: ```python >>> from transformers import VitMatteImageProcessor, VitMatteForImageMatting >>> import torch >>> from PIL import Image >>> from huggingface_hub import hf_hub_download >>> processor = VitMatteImageProcessor.from_pretrained("hustvl/vitmatte-small-composition-1k") >>> model = VitMatteForImageMatting.from_pretrained("hustvl/vitmatte-small-composition-1k") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/image-matting-fixtures", filename="image.png", repo_type="dataset" ... ) >>> image = Image.open(filepath).convert("RGB") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/image-matting-fixtures", filename="trimap.png", repo_type="dataset" ... ) >>> trimap = Image.open(filepath).convert("L") >>> # prepare image + trimap for the model >>> inputs = processor(images=image, trimaps=trimap, return_tensors="pt") >>> with torch.no_grad(): ... alphas = model(**inputs).alphas >>> print(alphas.shape) torch.Size([1, 1, 640, 960]) ```N)rcrbrSzTraining is not yet supportedr)rrrr) r#Zuse_return_dictrcrbr^Zforward_with_filtered_kwargsZ feature_mapsr_NotImplementedErrorrrr) r%rrbrcrdreoutputsrPrroutputrrrr=s.,  zVitMatteForImageMatting.forward)NNNNN)rrrr4r VITMATTE_INPUTS_DOCSTRINGformatr r_CONFIG_FOR_DOCrrZTensorboolr=r?rrr:rr\s  r\)"r dataclassesrtypingrrrrrZmodeling_utilsrutilsr r r r Zutils.backbone_utilsr Zconfiguration_vitmatterZ&VITMATTE_PRETRAINED_MODEL_ARCHIVE_LISTrkrrModuler,r@rLrRrWZVITMATTE_START_DOCSTRINGrir\rrrrs6      )