U ,-eUC@sdZddlZddlmZddlmZddlmZddlm Z dd l m Z e e Zd d iZGd d d eZGdddeZGdddeZdS)z! InstructBLIP model configurationN)Union)PretrainedConfig)!MODEL_FOR_CAUSAL_LM_MAPPING_NAMES)logging)CONFIG_MAPPINGz Salesforce/instruct-blip-flan-t5zPhttps://huggingface.co/Salesforce/instruct-blip-flan-t5/resolve/main/config.jsonc sDeZdZdZdZdfdd Zeeee j fddddZ Z S)InstructBlipVisionConfigaW This is the configuration class to store the configuration of a [`InstructBlipVisionModel`]. It is used to instantiate a InstructBLIP vision encoder according to the specified arguments, defining the model architecture. Instantiating a configuration defaults will yield a similar configuration to that of the InstructBLIP [Salesforce/instruct-blip-flan-t5](https://huggingface.co/Salesforce/instruct-blip-flan-t5) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: hidden_size (`int`, *optional*, defaults to 1408): Dimensionality of the encoder layers and the pooler layer. intermediate_size (`int`, *optional*, defaults to 6144): Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. num_hidden_layers (`int`, *optional*, defaults to 39): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer encoder. image_size (`int`, *optional*, defaults to 224): The size (resolution) of each image. patch_size (`int`, *optional*, defaults to 14): The size (resolution) of each patch. hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"selu"` and `"gelu_new"` ``"gelu"` are supported. to 1e-5): The epsilon used by the layer normalization layers. layer_norm_eps (`float`, *optional*, defaults to 1e-6): The epsilon used by the layer normalization layers. attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. initializer_range (`float`, *optional*, defaults to 1e-10): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. qkv_bias (`bool`, *optional*, defaults to `True`): Whether to add a bias to the queries and values in the self-attention layers. Example: ```python >>> from transformers import InstructBlipVisionConfig, InstructBlipVisionModel >>> # Initializing a InstructBlipVisionConfig with Salesforce/instruct-blip-flan-t5 style configuration >>> configuration = InstructBlipVisionConfig() >>> # Initializing a InstructBlipVisionModel (with random weights) from the Salesforce/instruct-blip-flan-t5 style configuration >>> model = InstructBlipVisionModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```Zinstructblip_vision_model'geluư>绽|=Tc  sTtjf| ||_||_||_||_||_||_| |_| |_ ||_ ||_ | |_ dS)N) super__init__ hidden_sizeintermediate_sizenum_hidden_layersnum_attention_heads patch_size image_sizeinitializer_rangeattention_dropoutlayer_norm_eps hidden_actqkv_bias) selfrrrrrrrrrrr kwargs __class__|/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/transformers/models/instructblip/configuration_instructblip.pyrVsz!InstructBlipVisionConfig.__init__rpretrained_model_name_or_pathreturncKs~|||j|f|\}}|ddkr2|d}d|krpt|drp|d|jkrptd|dd|jd|j|f|S)N model_type instructblip vision_configYou are using a model of type to instantiate a model of type N. This is not supported for all configurations of models and can yield errors.Z_set_token_in_kwargsZget_config_dictgethasattrr*loggerwarning from_dictclsr(r"Z config_dictr%r%r&from_pretrainedss  z(InstructBlipVisionConfig.from_pretrained) r r r r rrrrrrT __name__ __module__ __qualname____doc__r*r classmethodrstrosPathLiker8 __classcell__r%r%r#r&r !s 2r csDeZdZdZdZdfdd Zeeee j fddddZ Z S)InstructBlipQFormerConfigaO This is the configuration class to store the configuration of a [`InstructBlipQFormerModel`]. It is used to instantiate a InstructBLIP Querying Transformer (Q-Former) model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the InstructBLIP [Salesforce/instruct-blip-flan-t5](https://huggingface.co/Salesforce/instruct-blip-flan-t5) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Note that [`InstructBlipQFormerModel`] is very similar to [`BertLMHeadModel`] with interleaved cross-attention. Args: vocab_size (`int`, *optional*, defaults to 30522): Vocabulary size of the Q-Former model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling the model. hidden_size (`int`, *optional*, defaults to 768): Dimensionality of the encoder layers and the pooler layer. num_hidden_layers (`int`, *optional*, defaults to 12): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 12): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 3072): Dimensionality of the "intermediate" (often named feed-forward) layer in the Transformer encoder. hidden_act (`str` or `Callable`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"silu"` and `"gelu_new"` are supported. hidden_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities. max_position_embeddings (`int`, *optional*, defaults to 512): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. layer_norm_eps (`float`, *optional*, defaults to 1e-12): The epsilon used by the layer normalization layers. position_embedding_type (`str`, *optional*, defaults to `"absolute"`): Type of position embedding. Choose one of `"absolute"`, `"relative_key"`, `"relative_key_query"`. For positional embeddings use `"absolute"`. For more information on `"relative_key"`, please refer to [Self-Attention with Relative Position Representations (Shaw et al.)](https://arxiv.org/abs/1803.02155). For more information on `"relative_key_query"`, please refer to *Method 4* in [Improve Transformer Models with Better Relative Position Embeddings (Huang et al.)](https://arxiv.org/abs/2009.13658). cross_attention_frequency (`int`, *optional*, defaults to 2): The frequency of adding cross-attention to the Transformer layers. encoder_hidden_size (`int`, *optional*, defaults to 1408): The hidden size of the hidden states for cross-attention. Examples: ```python >>> from transformers import InstructBlipQFormerConfig, InstructBlipQFormerModel >>> # Initializing a InstructBLIP Salesforce/instruct-blip-flan-t5 style configuration >>> configuration = InstructBlipQFormerConfig() >>> # Initializing a model (with random weights) from the Salesforce/instruct-blip-flan-t5 style configuration >>> model = InstructBlipQFormerModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```Zinstructblip_qformer:w r皙?{Gz?-q=rabsoluterr c sntjfd| i|||_||_||_||_||_||_||_||_ | |_ | |_ | |_ | |_ ||_||_dS)N pad_token_id)rr vocab_sizerrrrrhidden_dropout_probattention_probs_dropout_probmax_position_embeddingsrrposition_embedding_typecross_attention_frequencyencoder_hidden_size)r!rNrrrrrrOrPrQrrrMrRrSrTr"r#r%r&rsz"InstructBlipQFormerConfig.__init__rr'cKs~|||j|f|\}}|ddkr2|d}d|krpt|drp|d|jkrptd|dd|jd|j|f|S)Nr*r+qformer_configr-r.r/r0r6r%r%r&r8s  z)InstructBlipQFormerConfig.from_pretrained)rDrErFrFrGrrHrHrIrJrKrrLrr r9r%r%r#r&rCs(<$rCcs<eZdZdZdZd fdd Zeeee ddd Z Z S) InstructBlipConfiga [`InstructBlipConfig`] is the configuration class to store the configuration of a [`InstructBlipForConditionalGeneration`]. It is used to instantiate a InstructBLIP model according to the specified arguments, defining the vision model, Q-Former model and language model configs. Instantiating a configuration with the defaults will yield a similar configuration to that of the InstructBLIP [Salesforce/instruct-blip-flan-t5](https://huggingface.co/Salesforce/instruct-blip-flan-t5) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vision_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`InstructBlipVisionConfig`]. qformer_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`InstructBlipQFormerConfig`]. text_config (`dict`, *optional*): Dictionary of configuration options used to initialize any [`PretrainedConfig`]. num_query_tokens (`int`, *optional*, defaults to 32): The number of query tokens passed through the Transformer. kwargs (*optional*): Dictionary of keyword arguments. Example: ```python >>> from transformers import ( ... InstructBlipVisionConfig, ... InstructBlipQFormerConfig, ... OPTConfig, ... InstructBlipConfig, ... InstructBlipForConditionalGeneration, ... ) >>> # Initializing a InstructBlipConfig with Salesforce/instruct-blip-flan-t5 style configuration >>> configuration = InstructBlipConfig() >>> # Initializing a InstructBlipForConditionalGeneration (with random weights) from the Salesforce/instruct-blip-flan-t5 style configuration >>> model = InstructBlipForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config >>> # We can also initialize a InstructBlipConfig from a InstructBlipVisionConfig, InstructBlipQFormerConfig and any PretrainedConfig >>> # Initializing InstructBLIP vision, InstructBLIP Q-Former and language model configurations >>> vision_config = InstructBlipVisionConfig() >>> qformer_config = InstructBlipQFormerConfig() >>> text_config = OPTConfig() >>> config = InstructBlipConfig.from_text_vision_configs(vision_config, qformer_config, text_config) ```r+N c stjf||dkr$i}td|dkr:i}td|dkrPi}tdtf||_tf||_d|krx|dnd}t|f||_ |j j |_ |j j |_ ||_ |jj |j_|j jtk|_d|_d|_dS)NzUvision_config is None. initializing the InstructBlipVisionConfig with default values.zWqformer_config is None. Initializing the InstructBlipQFormerConfig with default values.zTtext_config is None. Initializing the text config with default values (`OPTConfig`).r*optg?rJ)rrr3infor r,rCrUr text_configZtie_word_embeddingsZis_encoder_decodernum_query_tokensrrTr*rZuse_decoder_only_language_modelZinitializer_factorr)r!r,rUrZr[r"Ztext_model_typer#r%r&r4s*        zInstructBlipConfig.__init__r,rUrZcKs"|f|||d|S)z Instantiate a [`InstructBlipConfig`] (or a derived class) from a InstructBLIP vision model, Q-Former and language model configurations. Returns: [`InstructBlipConfig`]: An instance of a configuration object r\)to_dict)r7r,rUrZr"r%r%r& from_vision_qformer_text_configsQsz3InstructBlipConfig.from_vision_qformer_text_configs)NNNrW) r:r;r<r=r*rr>r rCrr^rBr%r%r#r&rVs5rV)r=r@typingrZconfiguration_utilsrZmodels.auto.modeling_autorutilsrautorZ get_loggerr:r3Z*INSTRUCTBLIP_PRETRAINED_CONFIG_ARCHIVE_MAPr rCrVr%r%r%r&s      ev