Multimodal Signal Analysis for Audio-Visual Speech Recognition

After a short introduction presenting our group and our main research topics, I will address the problem of audio-visual speech recognition, i.e. a typical example of multimodal signal analysis, when we want to extract and exploit information coming from two different but complementary signals: an audio and a video channel. We will discuss two important aspects of this analysis. We will first present a new feature extraction algorithm based in information theoretical principles, and show its performances, compared to other classical approaches, in our multimodal context. Then we will discuss multimodal information fusion, i.e. how to combine information from those two channels for optimal classification.
Jean-Philippe Thiran was born in Namur, Belgium, in 1970. He received the Elect. Eng. and Ph.D. degrees from the Universite catholique de Louvain (UCL), Louvain-la-Neuve, Belgium, in 1993 and 1997, respectively. He joined the Signal Processing Laboratory (LTS) of the Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland, in February 1998 as a Senior Lecturer. Since January 2004, he has been an Assistant Professor, responsible for the Image Analysis Group. His current scientific interests include image segmentation, prior knowledge integration in image analysis, partial differential equations and variational methods in image analysis, multimodal signal processing, medical image analysis, including multimodal image registration, segmentation, computer-assisted surgery, and diffusion MRI. He is author or co-author of two 4 book chapters, 69 journal papers, and some 130 peer-reviewed papers published in proceedings of international conferences. He holds four international patents. Dr. Thiran was Co-Editor-in-Chief of Signal Processing (published by Elsevier Science) from 2001 to 2005. He is currently an Associate Editor of the International Journal of Image and Video Processing (published by Hindawi), and member of the Editorial Board of Signal, Image and Video Processing (published by Springer). He was the General Chairman of the 2008 European Signal Processing Conference (EUSIPCO 2008). He is a senior member of the IEEE, and member of the MLSP and IVMSP technical committees of the IEEE Signal Processing Society.