U ,-e@sBdZddlmZddlmZeeZddiZGdddeZ dS) z CPMAnt model configuration)PretrainedConfig)loggingzopenbmb/cpm-ant-10bz@https://huggingface.co/openbmb/cpm-ant-10b/blob/main/config.jsoncsHeZdZdZdZdeeeeeeeeeeeeeeedfdd ZZ S) CpmAntConfiga This is the configuration class to store the configuration of a [`CpmAntModel`]. It is used to instantiate an CPMAnt model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the CPMAnt [openbmb/cpm-ant-10b](https://huggingface.co/openbmb/cpm-ant-10b) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 30720): Vocabulary size of the CPMAnt model. Defines the number of different tokens that can be represented by the `input` passed when calling [`CpmAntModel`]. hidden_size (`int`, *optional*, defaults to 4096): Dimension of the encoder layers. num_attention_heads (`int`, *optional*, defaults to 32): Number of attention heads in the Transformer encoder. dim_head (`int`, *optional*, defaults to 128): Dimension of attention heads for each attention layer in the Transformer encoder. dim_ff (`int`, *optional*, defaults to 10240): Dimension of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. num_hidden_layers (`int`, *optional*, defaults to 48): Number of layers of the Transformer encoder. dropout_p (`float`, *optional*, defaults to 0.1): The dropout probabilitiy for all fully connected layers in the embeddings, encoder. position_bias_num_buckets (`int`, *optional*, defaults to 512): The number of position_bias buckets. position_bias_max_distance (`int`, *optional*, defaults to 2048): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). eps (`float`, *optional*, defaults to 1e-6): The epsilon used by the layer normalization layers. prompt_types (`int`, *optional*, defaults to 32): The type of prompt. prompt_length (`int`, *optional*, defaults to 32): The length of prompt. segment_types (`int`, *optional*, defaults to 32): The type of segment. use_cache (`bool`, *optional*, defaults to `True`): Whether to use cache. init_std (`float`, *optional*, defaults to 1.0): Initialize parameters with std = init_std. Example: ```python >>> from transformers import CpmAntModel, CpmAntConfig >>> # Initializing a CPMAnt cpm-ant-10b style configuration >>> configuration = CpmAntConfig() >>> # Initializing a model from the cpm-ant-10b style configuration >>> model = CpmAntModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```Zcpmantx (0ư>?T) vocab_size hidden_sizenum_attention_headsdim_headdim_ffnum_hidden_layers dropout_pposition_bias_num_bucketsposition_bias_max_distanceepsinit_std prompt_types prompt_length segment_types use_cachec sltjf|| |_| |_||_||_||_||_||_||_ ||_ | |_ ||_ | |_ ||_||_| |_dS)N)super__init__rrrrrrrrrrrrrrr)selfrrrrrrrrrrrrrrrkwargs __class__p/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/transformers/models/cpmant/configuration_cpmant.pyr Ys zCpmAntConfig.__init__)rrrrr r r r r rrrrrT) __name__ __module__ __qualname____doc__Z model_typeintfloatboolr __classcell__r%r%r#r&rsD9rN) r*Zconfiguration_utilsrutilsrZ get_loggerr'loggerZ$CPMANT_PRETRAINED_CONFIG_ARCHIVE_MAPrr%r%r%r&s