MUHCI – “Multi-modal Human-Computer Interaction”

MUHCI – “Multi-modal Human-Computer Interaction”

Period: May 2000 - August 2004 Type: European Research Status: Completed
3 Aug 2017


MUHCI focuses on human-centered interfaces. Our approach is to develop a human-like computer that can understand the behavior of human beings, can express its emotions to us, and can communicate and interact with us in natural ways. Like a normal human, such a smart computer should have functional blocks: sensors, brain, and actuators. In the project, a number of challenging, closely coupled analysis tasks is identified, namely, (a) robustifying face analysis (face localisation, facial feature extraction, facial feature tracking); (b) face verification; (c) facial expression and face gesture recognition; (d) hand gesture recognition; (e) automatic interpretation of user’s behavior from cursor movements; (f) audio-visual speech recognition; (g) audio-visual speech synthesis; (h) improving 3-D face modeling; (i) facial animation providing higher quality of emotions. The aforementioned analysis tasks correspond to the “sensors+brain” part.

Synthesis tasks, such as the development of talking heads and active browsers, are addressed, that correspond to the “brain+actuators” part. Other applications, closely related to the above defined tasks, are also promoted, i.e., (a) developing, extending and improving several techniques for building databases of faces and bodies (cloning from photographs, morphing, generating populations, manual editing), and publishing facial animation content on the Internet; (b) model-based natural-synthetic coding.

Human-computer bi-directional interaction needs a mixture of experts coming from different disciplines, such as, image processing, computer vision, speech processing, pattern recognition, computer graphics, communications, linguistics, etc. It provides a fertile ground for exchanging ideas and cooperation that will greatly facilitate the proper treatment of the aforementioned complicated problems. A work plan is set up to provide solutions that improve the state-of-the-art techniques used at each problem. The quality of the solutions that will be given to each research topic depends strongly on the synergy and the cooperation of experts from different fields.


Aristotle University of Thessaloniki

Helsinki University of Technology

Linköpings universitet

MIRALab, University of Geneva

Ruhr-Universitaet Bochum

Tampere University of Technology

Umea University

W Interactive SARL