U 9%e@G@sdZddlZddlmZmZmZer$ddlmZddlm Z e e Z ddiZ Gd d d eZGd d d eZGd ddeZdS)z ALIGN model configurationN) TYPE_CHECKINGListUnion)PretrainedConfig)loggingzkakaobrain/align-basezEhttps://huggingface.co/kakaobrain/align-base/resolve/main/config.jsoncsDeZdZdZdZdfdd Zeeee j fddddZ Z S)AlignTextConfigak This is the configuration class to store the configuration of a [`AlignTextModel`]. It is used to instantiate a ALIGN text encoder according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the text encoder of the ALIGN [kakaobrain/align-base](https://huggingface.co/kakaobrain/align-base) architecture. The default values here are copied from BERT. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 30522): Vocabulary size of the Align Text model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`AlignTextModel`]. hidden_size (`int`, *optional*, defaults to 768): Dimensionality of the encoder layers and the pooler layer. num_hidden_layers (`int`, *optional*, defaults to 12): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 12): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 3072): Dimensionality of the "intermediate" (often named feed-forward) layer in the Transformer encoder. hidden_act (`str` or `Callable`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"silu"` and `"gelu_new"` are supported. hidden_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities. max_position_embeddings (`int`, *optional*, defaults to 512): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). type_vocab_size (`int`, *optional*, defaults to 2): The vocabulary size of the `token_type_ids` passed when calling [`AlignTextModel`]. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. layer_norm_eps (`float`, *optional*, defaults to 1e-12): The epsilon used by the layer normalization layers. position_embedding_type (`str`, *optional*, defaults to `"absolute"`): Type of position embedding. Choose one of `"absolute"`, `"relative_key"`, `"relative_key_query"`. For positional embeddings use `"absolute"`. For more information on `"relative_key"`, please refer to [Self-Attention with Relative Position Representations (Shaw et al.)](https://arxiv.org/abs/1803.02155). For more information on `"relative_key_query"`, please refer to *Method 4* in [Improve Transformer Models with Better Relative Position Embeddings (Huang et al.)](https://arxiv.org/abs/2009.13658). use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). Only relevant if `config.is_decoder=True`. pad_token_id (`int`, *optional*, defaults to 0) Padding token id. Example: ```python >>> from transformers import AlignTextConfig, AlignTextModel >>> # Initializing a AlignTextConfig with kakaobrain/align-base style configuration >>> configuration = AlignTextConfig() >>> # Initializing a AlignTextModel (with random weights) from the kakaobrain/align-base style configuration >>> model = AlignTextModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```Zalign_text_model:w gelu皙?{Gz?-q=rabsoluteTc sltjf|||_||_||_||_||_||_||_||_ | |_ | |_ | |_ | |_ ||_||_| |_dS)N)super__init__ vocab_size hidden_sizenum_hidden_layersnum_attention_heads hidden_actintermediate_sizehidden_dropout_probattention_probs_dropout_probmax_position_embeddingstype_vocab_sizeinitializer_rangelayer_norm_epsposition_embedding_type use_cache pad_token_id)selfrrrrrrrrrrr r!r$r"r#kwargs __class__l/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/transformers/models/align/configuration_align.pyrfs zAlignTextConfig.__init__rpretrained_model_name_or_pathreturncKs~|||j|f|\}}|ddkr2|d}d|krpt|drp|d|jkrptd|dd|jd|j|f|S)N model_typealign text_configYou are using a model of type to instantiate a model of type N. This is not supported for all configurations of models and can yield errors.Z_set_token_in_kwargsZget_config_dictgethasattrr.loggerwarning from_dictclsr,r&Z config_dictr)r)r*from_pretraineds  zAlignTextConfig.from_pretrained)r r r r r r rrrrrrrrT) __name__ __module__ __qualname____doc__r.r classmethodrstrosPathLiker< __classcell__r)r)r'r*r#s(@%rcseZdZdZdZddddddddddddgd d d d d ddgd d d d dddggdddddddgdddddddgdddddddgddddddddfeeeeeeeeeeeeeeeeeeeeeeeeeeedfdd Z e e ee j fd d!d"d#ZZS)$AlignVisionConfiga  This is the configuration class to store the configuration of a [`AlignVisionModel`]. It is used to instantiate a ALIGN vision encoder according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the vision encoder of the ALIGN [kakaobrain/align-base](https://huggingface.co/kakaobrain/align-base) architecture. The default values are copied from EfficientNet (efficientnet-b7) Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: num_channels (`int`, *optional*, defaults to 3): The number of input channels. image_size (`int`, *optional*, defaults to 600): The input image size. width_coefficient (`float`, *optional*, defaults to 2.0): Scaling coefficient for network width at each stage. depth_coefficient (`float`, *optional*, defaults to 3.1): Scaling coefficient for network depth at each stage. depth_divisor `int`, *optional*, defaults to 8): A unit of network width. kernel_sizes (`List[int]`, *optional*, defaults to `[3, 3, 5, 3, 5, 5, 3]`): List of kernel sizes to be used in each block. in_channels (`List[int]`, *optional*, defaults to `[32, 16, 24, 40, 80, 112, 192]`): List of input channel sizes to be used in each block for convolutional layers. out_channels (`List[int]`, *optional*, defaults to `[16, 24, 40, 80, 112, 192, 320]`): List of output channel sizes to be used in each block for convolutional layers. depthwise_padding (`List[int]`, *optional*, defaults to `[]`): List of block indices with square padding. strides (`List[int]`, *optional*, defaults to `[1, 2, 2, 2, 1, 2, 1]`): List of stride sizes to be used in each block for convolutional layers. num_block_repeats (`List[int]`, *optional*, defaults to `[1, 2, 2, 3, 3, 4, 1]`): List of the number of times each block is to repeated. expand_ratios (`List[int]`, *optional*, defaults to `[1, 6, 6, 6, 6, 6, 6]`): List of scaling coefficient of each block. squeeze_expansion_ratio (`float`, *optional*, defaults to 0.25): Squeeze expansion ratio. hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in each block. If string, `"gelu"`, `"relu"`, `"selu", `"gelu_new"`, `"silu"` and `"mish"` are supported. hiddem_dim (`int`, *optional*, defaults to 1280): The hidden dimension of the layer before the classification head. pooling_type (`str` or `function`, *optional*, defaults to `"mean"`): Type of final pooling to be applied before the dense classification head. Available options are [`"mean"`, `"max"`] initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. batch_norm_eps (`float`, *optional*, defaults to 1e-3): The epsilon used by the batch normalization layers. batch_norm_momentum (`float`, *optional*, defaults to 0.99): The momentum used by the batch normalization layers. drop_connect_rate (`float`, *optional*, defaults to 0.2): The drop rate for skip connections. Example: ```python >>> from transformers import AlignVisionConfig, AlignVisionModel >>> # Initializing a AlignVisionConfig with kakaobrain/align-base style configuration >>> configuration = AlignVisionConfig() >>> # Initializing a AlignVisionModel (with random weights) from the kakaobrain/align-base style configuration >>> model = AlignVisionModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```Zalign_vision_modelriXg@g@ (Ppi@rg?Zswishi ZmeanrgMbP?gGz?g?) num_channels image_sizewidth_coefficientdepth_coefficient depth_divisor kernel_sizes in_channels out_channelsdepthwise_paddingstridesnum_block_repeats expand_ratiossqueeze_expansion_ratior hidden_dim pooling_typer batch_norm_epsbatch_norm_momentumdrop_connect_ratec stjf|||_||_||_||_||_||_||_||_ | |_ | |_ | |_ | |_ | |_||_||_||_||_||_||_||_t| d|_dS)NrQ)rrrSrTrUrVrWrXrYrZr[r\r]r^r_rr`rar rbrcrdsumr)r%rSrTrUrVrWrXrYrZr[r\r]r^r_rr`rar rbrcrdr&r'r)r*rs,zAlignVisionConfig.__init__rr+cKs~|||j|f|\}}|ddkr2|d}d|krpt|drp|d|jkrptd|dd|jd|j|f|S)Nr.r/ vision_configr1r2r3r4r:r)r)r*r<s  z!AlignVisionConfig.from_pretrained)r=r>r?r@r.intfloatrrBrrArrCrDr<rEr)r)r'r*rFs\E0rFcs:eZdZdZdZd fdd Zeeed d d Z Z S) AlignConfiga [`AlignConfig`] is the configuration class to store the configuration of a [`AlignModel`]. It is used to instantiate a ALIGN model according to the specified arguments, defining the text model and vision model configs. Instantiating a configuration with the defaults will yield a similar configuration to that of the ALIGN [kakaobrain/align-base](https://huggingface.co/kakaobrain/align-base) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: text_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`AlignTextConfig`]. vision_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`AlignVisionConfig`]. projection_dim (`int`, *optional*, defaults to 640): Dimentionality of text and vision projection layers. temperature_init_value (`float`, *optional*, defaults to 1.0): The inital value of the *temperature* paramter. Default is used as per the original ALIGN implementation. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. kwargs (*optional*): Dictionary of keyword arguments. Example: ```python >>> from transformers import AlignConfig, AlignModel >>> # Initializing a AlignConfig with kakaobrain/align-base style configuration >>> configuration = AlignConfig() >>> # Initializing a AlignModel (with random weights) from the kakaobrain/align-base style configuration >>> model = AlignModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config >>> # We can also initialize a AlignConfig from a AlignTextConfig and a AlignVisionConfig >>> from transformers import AlignTextConfig, AlignVisionConfig >>> # Initializing ALIGN Text and Vision configurations >>> config_text = AlignTextConfig() >>> config_vision = AlignVisionConfig() >>> config = AlignConfig.from_text_vision_configs(config_text, config_vision) ```r/N?rc shtjf||dkr$i}td|dkr:i}tdtf||_tf||_||_||_ ||_ dS)NzJtext_config is None. Initializing the AlignTextConfig with default values.zNvision_config is None. Initializing the AlignVisionConfig with default values.) rrr7inforr0rFrfprojection_dimtemperature_init_valuer )r%r0rfrmrnr r&r'r)r*r[s     zAlignConfig.__init__r0rfcKs|f||d|S)z Instantiate a [`AlignConfig`] (or a derived class) from align text model configuration and align vision model configuration. Returns: [`AlignConfig`]: An instance of a configuration object ro)to_dict)r;r0rfr&r)r)r*from_text_vision_configsus z$AlignConfig.from_text_vision_configs)NNrjrkr) r=r>r?r@r.rrArrFrqrEr)r)r'r*ri)s/ri)r@rCtypingrrrZconfiguration_utilsrutilsrZ get_loggerr=r7Z#ALIGN_PRETRAINED_CONFIG_ARCHIVE_MAPrrFrir)r)r)r*s   {