U 9%e!3@sddlZddlmZmZmZmZddlmZmZm Z m Z ddl m Z m Z erVddlZe rdddlZe eZGdddZee d Gd d d e ZdS) N)AnyDictListUnion)add_end_docstringsis_tf_availableis_torch_availablelogging)PIPELINE_INIT_ARGSPipelinec@seZdZdZd$eeeeeeffej dddZ ddZ eeefdd d Z d%ee d d dZedddZddZddZddZddZddZddZddZed d!Zed"d#ZdS)& Conversationa Utility class containing a conversation and its history. This class is meant to be used as an input to the [`ConversationalPipeline`]. The conversation contains several utility functions to manage the addition of new user inputs and generated model responses. Arguments: messages (Union[str, List[Dict[str, str]]], *optional*): The initial messages to start the conversation, either a string, or a list of dicts containing "role" and "content" keys. If a string is passed, it is interpreted as a single message with the "user" role. conversation_id (`uuid.UUID`, *optional*): Unique identifier for the conversation. If not provided, a random UUID4 id will be assigned to the conversation. Usage: ```python conversation = Conversation("Going to the movies tonight - any suggestions?") conversation.add_message({"role": "assistant", "content": "The Big lebowski."}) conversation.add_message({"role": "user", "content": "Is it good?"}) ```N)messagesconversation_idc Ks|s t}|dkr<|dd}|dk r6d|dg}qRg}nt|trRd|dg}|dd}|dd}|dk r|dkrtd|dk rg}|dkrg}ttt|t|gD]D}|t|kr| d||d|t|kr| d||dq||}||_||_ dS)Ntextuserrolecontentgenerated_responsespast_user_inputsz>generated_responses cannot be passed without past_user_inputs! assistant) uuiduuid4pop isinstancestr ValueErrorrangemaxlenappendr) selfrrZdeprecated_kwargsrrrZlegacy_messagesir%d/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/transformers/pipelines/conversational.py__init__(s2        zConversation.__init__cCs&t|tsdS|j|jkp$|j|jkS)NF)rrrr)r#otherr%r%r&__eq__Ls zConversation.__eq__)messagecCs@t|ddhkstd|ddkr0td|j|dS)Nrrz6Message should contain only 'role' and 'content' keys!)rrsystemzBOnly 'user', 'assistant' and 'system' roles are supported for now!)setkeysrrr"r#r*r%r%r& add_messageQs  zConversation.add_messageF)r overwritecCst|dkrp|dddkrp|rNtd|ddd|d||dd<qtd|ddd |d n|jd|d d S) a3 Add a user input to the conversation for the next round. This is a legacy method that assumes that inputs must alternate user/assistant/user/assistant, and so will not add multiple user messages in succession. We recommend just using `add_message` with role "user" instead. rrrz8User input added while unprocessed input was existing: "rz" was overwritten with: "z".z" new input ignored: "z>". Set `overwrite` to True to overwrite unprocessed user inputrN)r!loggerwarningrr")r#rr0r%r%r&add_user_inputXszConversation.add_user_input)responsecCs|jd|ddS)zr This is a legacy method. We recommend just using `add_message` with an appropriate role instead. rrN)rr")r#r5r%r%r&append_responsemszConversation.append_responsecCsdS)z This is a legacy method that no longer has any effect, as the Conversation no longer distinguishes between processed and unprocessed user input. Nr%r#r%r%r&mark_processedsszConversation.mark_processedccs|jD] }|VqdSNrr.r%r%r&__iter__zs zConversation.__iter__cCs |j|Sr9r:)r#itemr%r%r& __getitem__~szConversation.__getitem__cCs||j|<dSr9r:)r#keyvaluer%r%r& __setitem__szConversation.__setitem__cCs t|jSr9)r!rr7r%r%r&__len__szConversation.__len__cCs:d|jd}|jD] }||dd|dd7}q|S)a Generates a string representation of the conversation. Returns: `str`: Example: Conversation id: 7d15686b-dc94-49f2-9c4b-c9eac6a1f114 user: Going to the movies tonight - any suggestions? bot: The Big Lebowski zConversation id:  rz: r)rr)r#outputr*r%r%r&__repr__s  zConversation.__repr__ccs&|jD]}|ddk|dfVqdS)Nrrrr:r.r%r%r& iter_textss zConversation.iter_textscCsdd|jDS)NcSs g|]}|ddkr|dqS)rrrr%.0r*r%r%r& s z1Conversation.past_user_inputs..r:r7r%r%r&rszConversation.past_user_inputscCsdd|jDS)NcSs g|]}|ddkr|dqS)rrrr%rFr%r%r&rHs z4Conversation.generated_responses..r:r7r%r%r&rsz Conversation.generated_responses)NN)F)__name__ __module__ __qualname____doc__rrrrrUUIDr'r)r/boolr4r6r8r;r=r@rArDrEpropertyrrr%r%r%r&rs, $ ra  min_length_for_response (`int`, *optional*, defaults to 32): The minimum length (in number of tokens) for a response. minimum_tokens (`int`, *optional*, defaults to 10): The minimum length of tokens to leave for a response. csxeZdZdZfddZdddZdeeeefdfd d Z dee e e fd d dZ dddZdddZZS)ConversationalPipelinea Multi-turn conversational pipeline. Example: ```python >>> from transformers import pipeline, Conversation >>> chatbot = pipeline(model="microsoft/DialoGPT-medium") >>> conversation = Conversation("Going to the movies tonight - any suggestions?") >>> conversation = chatbot(conversation) >>> conversation.generated_responses[-1] 'The Big Lebowski' >>> conversation.add_user_input("Is it an action movie?") >>> conversation = chatbot(conversation) >>> conversation.generated_responses[-1] "It's a comedy." ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This conversational pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"conversational"`. The models that this pipeline can use are models that have been fine-tuned on a multi-turn conversational task, currently: *'microsoft/DialoGPT-small'*, *'microsoft/DialoGPT-medium'*, *'microsoft/DialoGPT-large'*. See the up-to-date list of available models on [huggingface.co/models](https://huggingface.co/models?filter=conversational). cs*tj|||jjdkr&|jj|j_dSr9)superr' tokenizerZ pad_token_idZ eos_tokenZ pad_token)r#argskwargs __class__r%r&r's zConversationalPipeline.__init__NcKshi}i}i}|dk r||d<|dk r,||d<d|kr@|d|d<|dk rP||d<|r^|||||fS)Nmin_length_for_responseminimum_tokens max_lengthclean_up_tokenization_spaces)update)r#rWrXrZgenerate_kwargsZpreprocess_paramsZforward_paramsZpostprocess_paramsr%r%r&_sanitize_parameterss  z+ConversationalPipeline._sanitize_parametersr) conversationsc s:tj|fd|i|}t|tr6t|dkr6|dS|S)a, Generate responses for the conversation(s) given as inputs. Args: conversations (a [`Conversation`] or a list of [`Conversation`]): Conversations to generate responses for. clean_up_tokenization_spaces (`bool`, *optional*, defaults to `False`): Whether or not to clean up the potential extra spaces in the text output. generate_kwargs: Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework [here](./model#generative-models)). Returns: [`Conversation`] or a list of [`Conversation`]: Conversation(s) with updated generated responses for those containing a new user input. num_workersr r)rQ__call__rlistr!)r#r^r_rToutputsrUr%r&r`szConversationalPipeline.__call__ ) conversationreturncCsD|j|}|jdkr$t|g}n|jdkr:t|g}||dS)Npttf) input_idsrd)rRZapply_chat_templateZ frameworktorchZ LongTensorrgZconstant)r#rdrWrhr%r%r& preprocesss     z!ConversationalPipeline.preprocess c Ks|d|jjj}|djd}|||krtd|d||d||}|ddd| df|d<d|kr|ddd| df|d<|d}||d<|jjf||}|jjj rd} n|} |dd| df|d S) NrYrhr z Conversation input is too long (z), trimming it to z> tokens. Consider increasing `max_length` to avoid truncation.Zattention_maskrd) output_idsrd) getmodelconfigrYshaper2r3rgenerateZis_encoder_decoder) r#Z model_inputsrXr\rYnZtrimrdrlZstart_positionr%r%r&_forwards"   zConversationalPipeline._forwardTcCs:|d}|jj|dd|d}|d}|d|d|S)NrlrT)Zskip_special_tokensrZrdrr)rRdecoder/)r#Z model_outputsrZrlZanswerrdr%r%r& postprocess&sz"ConversationalPipeline.postprocess)NNN)r)rc)rk)T)rIrJrKrLr'r]rrrr`rrrrjrsru __classcell__r%r%rUr&rPs     rP)rtypingrrrrutilsrrr r baser r Z tensorflowrgriZ get_loggerrIr2rrPr%r%r%r&s