End-to-End Spoken Language Understanding on real world tasks

Title

Mentored by

Mohamed Morchid, Associate Professor

Location

Avignon University, LIA, France

Global Context

Project: The internship is strongly related to ANR AISSPER for end-to-end spoken language understanding in different levels of the document content. Web sites:

• ANR

• ResearchGate

Period: February 2020 to July 2020

Salary: 577.50€ p.m

Aims and objectives

Neural networks based algorithms are nowadays employed in a massive set of real-world related systems and applications. This internship focuses on Natural Language Processing (NLP) tasks such as Spoken Language Understanding (SLU). Among SLU based models, end-to-end (EtE) neural systems are promising in regard to the results observed already with EtE Automatic Speech Recognition (ASR) with neural based systems. A main drawback of hitherto proposed neural based SLU systems, is related to the need of a two-step process to successively extract high dimensional representation of the relevant content from the spoken signal in a homogeneous feature hidden space (ASR block) alongside to interpret these abstract features as understandable discussed subjects, mentions or intents contained in the spoken dialogue (Natural LU block). Therefore, the errors observed during these two steps are hardly located and characterized, and are propagated throughout the neural SLU chain block. These biases may therefore massively affect neural SLU-based systems performances.

The objectives of this internship is first to study state-of-the-art neural SLU systems and EtE neural SLU models [1][2][3]. The intern will propose novel neural based architectures that consider both high level features from hidden spaces and the language understanding process in a unified neural based algorithm. Quaternion neural networks among other neural based models can be employed [4].

This internship can be followed by a Ph.D. within the Avignon University.

References

[1] Tomashenko, N., Caubrière, A., & Estève, Y. Investigating adaptation and transfer learning for end-to-end spoken language understanding from speech. Interspeech 2019.

[2] Caubrière, A., Tomashenko, N., Laurent, A., Morin, E., Camelin, N., & Estève, Y. (2019). Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portability. arXiv preprint arXiv:1906.07601. Interspeech 2019.

[3] Lugosch, L., Ravanelli, M., Ignoto, P., Tomar, V. S., & Bengio, Y.. Speech Model Pre-training for End-to-End Spoken Language Understanding, Interspeech 2019.

[4] Parcollet Titouan, Morchid Mohamed, Bost Xavier, Linares Georges, De Mori Renato. Real to H-space Autoencoders for Theme Identification in Telephone Conversations, IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020.

Related topics: