Multimodal Emotion Recognition

Human communication is modulated by many affective phenomena including emotions, moods, interpersonal stances, sentiment, and personality. The goal of this project is to is to develop algorithms and computational representations able to automatically recognize human emotional states from speech and visual behaviors. This project focuses on acoustic and visual behaviors, including facial appearance and expressions, eye gaze, head gestures, prosodic cues and voice quality. We will develop recognition models able to predict emotions over time (from audio-visual sequences) and study approaches to reduce the effect of person-specific expressions (e.g., idiosyncrasy).