LIPPS – A Virtual Teacher for Speechreading Based on a Dialog-Controlled Talking-Head

People with decreasing hearing ability are more dependent on alternative personal communication channels. To read the visible articulator movements of the conversation partner is one possible solution for understanding verbal statements. After giving an introduction to the principals of speech, visual speech & speechreading, we demonstrate the conception and implementation of a language laboratory for speechreading that is meant as an effective audio-visual complement and extension of classroom teaching.

Further we show results of evaluation experiments and explain the functionality and interaction of the modules, the interaction between student, teacher, and virtual teacher and the structure of the pedagogical approach to teach speechreading with the help of a virtual teacher.