Artificial Social Intelligence
Human face-to-face communication is a little like a dance: participants continuously adjust their behaviors based on their interlocutor’s speech, gestures, and facial expressions during social interaction. The actual sophistication of human communication comes to the fore when we try to create computers that are able to understand and participate, however crudely, in these types of social interactions. It is then when the complexity of the process and, accordingly, the enormity of the challenge become apparent. Our group’s research takes up this challenge: building the computational foundations to enable computers with the abilities to analyze, recognize, and predict subtle human communicative behaviors during social interactions.
Our research vision for human communication dynamics modeling is defined by four key computational challenges: behavioral dynamics to model the appearance and temporal variations of individual communicative behaviors and their effects on perceived meanings; multimodal dynamics to model the interdependence between different communicative channels including visual gestures and expressions, language and acoustic signals; interpersonal dynamics to model the social and conversational influence between participants during dyadic or small-group interactions (i.e., micro-level); and societal dynamics to model the cultural and behavioral changes in a larger groups (i.e., meso-level) or in different societies (i.e, macro-level).
MultiComp Lab’s research projects in human communication dynamics span multiple levels of analysis. At the first level, we build algorithms for sensing human visual and vocal behavior cues such as facial expressions, eye gaze, and head gestures. Robust and accurate detection of facial landmark is an important pre-processing step for many of these visual sensing technologies. At the next two levels, we perform recognition and interpretation of human affective and emotional states by integrating observations over time. This temporal information is also analyzed in the context of the social interaction, either dyadic, in group, or as a society. Our research also studies computational modeling of cognitive and reasoning processes, especially when building technologies for education and learning science. It is one of the most exciting research program, one that fosters novel interdisciplinary collaborations and creates unique intellectual challenges that go well beyond computing into other areas of academia and have a clear public impact.