Date: Friday, December 19, at 2 p.m.
Lieu: Ada amphitheater at CERI.
The defense will be presented in English.
Title: Open-Domain Conversational Agents with Transformer-Based Language Models: Toward Multilingualism and Personality
The jury will be composed of:
– Lina M. Rojas-Barahona , HDR, Orange Innovation, Reviewer
– Didier Schwab, Professor, LIG/GETALP, Université de Grenoble, Reviewer
– Sophie Rosset, Professor, LISN, Université Paris Saclay, Examiner
– David Traum, Professor, ICT, University of Southern California, Examiner
– Bassam Jabaian, Associate Professor, LIA, Université d’Avignon, Thesis Co-supervisor
– Fabrice Lefèvre, Professor, LIA, Université d’Avignon, Thesis Director
Abstract: Open-Domain Dialogue (ODD) systems are conversational agents designed for natural and open-ended human interaction. The proliferation of Conversational AI tools like ChatGPT has recently reshaped user expectations; beyond grammatical correctness, users now demand agents that demonstrate contextual understanding, cultural awareness, distinct personality, factual consistency, and other human-like conversational abilities. Despite the impressive progress, ODD systems development has long faced key limitations including strong linguistic bias towards English and Chinese, and the Open-Domain Paradox (ODP) (Skantze and Doğruöz, 2023), which constrains genuine conversational diversity and openness. This dissertation tackles these challenges by exploring multilingual and personality-centric strategies for building controllable and culturally adaptive ODD systems using Transformer-based Language Models. The research progresses along the following complementary axes.
First, we investigate multilingual portability using Machine Translation-based approaches, comparing two configurations: “Train on Target” which involves translating source-language data to fine-tune target-language models and “Test on Source” which employs inference-time translation with source-language models. Our findings indicate that multilingual models like BLOOM display decent robustness to translation artifacts, though their performance remains below source-language baselines.
Second, to overcome the inherent limitations of Machine Translation, we introduce the Multilingual Open-domain Unnatural Dialogues (MOUD) dataset, a multilingual, culturally-nuanced synthetic dialogue corpus generated with Instructions-Following Large Language Models (LLMs). MOUD converts human-centric data collection guidelines into structured prompts that embed language-specific knowledge such as named entities, and folk psychology, thereby enriching linguistic diversity and mitigating the ODP.
Then, we delve into personality modeling and human-likeness in dialogue. We develop structured role-play prompting techniques that simulate human conversational behaviors and propose a novel vector-based personality representation grounded in the Big Five (OCEAN) traits. These frameworks allowed for precise control over dialogue style and personality expression. Empirical assessments demonstrate significant improvements in perceived humanness, coherence, and engagement, while also highlighting a dependency on model-specific emergent abilities.
To mitigate this dependency and capitalize on MOUD, we explore internal and architectural solutions for cross-lingual transfer. We adapt adapter-based methods (MAD-X, Pfeiffer et al., 2020) to ODD task as a baseline and propose a novel framework: Sem2Seq-ns, a hybrid approach combining language-agnostic semantic representations with neuron specialization within Transformer layers. Although comprehensive results are pending, we anticipate performance improvements, particularly for low-resource languages not seen during fine-tuning.