Next SLG meeting will take place on 03/15/2024, from 10 AM to 11 AM.
We will host Antoine Caubrière from the company Orange, who will present his recent work
Title: Representation of Multilingual Speech through Self-Supervised Learning in an Exclusively Sub-Saharan Context.
Abstract: The Orange group operates in over a dozen sub-Saharan African countries with the ambition of offering services tailored to the needs of clients in this region. To provide localized and accessible services to digitally underserved and low-literate individuals, Orange is investing in the development of voice-based conversational agents to inform and assist its clients and employees.
The implementation of such a service requires, first and foremost, a technological component for speech recognition and understanding.
The strong linguistic diversity of the African continent, coupled with the challenges of limited annotated data, poses one of the challenges in implementing speech processing technology for these languages.
One potential solution could be the utilization of self-supervised learning techniques. Leveraging this type of learning enables the training of a speech representation extractor capable of capturing rich features. This approach utilizes a large quantity of unlabeled data for pre-training a model before fine-tuning it for specific tasks. While numerous self-supervised models are shared within the scientific community, unfortunately, sub-Saharan languages remain significantly underrepresented in their multilingual training sets (MMS, XLSR, etc.).
Building upon this observation, our recent work has focused on developing a self-supervised system exclusively using sub-Saharan language data. We have collected online data available in 21 languages and dialects for self-supervised learning of speech representation. By relying on the sub-Saharan portion of the FLEURS corpus, we derived models for speech recognition tasks.