Automatic prediction of psychosis symptoms from facial expressions

Psychotic disorders, including schizophrenia and bipolar disorder, affect the perception of individuals and how they express themselves. Symptoms typically include hallucinations, blunted affect, and anxiety. Compared to depression, which exhibits mostly inactive symptoms (reduction in normal function, such as blunted affect and emotional withdrawal), psychotic disorders are also described by active symptoms (increase in normal function, such as grandiosity and hallucinations).

Psychotic disorders affect how individuals speak and how they express themselves with facial expressions. Such differences in speech and expression may be difficult for humans to assess objectively, but they can be captured by computational methods. These differences bring the opportunity to support clinicians in assessing symptoms and allow for better decision-making. Importantly, the goal of this work is not to replace clinicians but rather to augment their abilities, to aid them more efficiently, and to begin to provide much-needed evidence to scaffold and help shape clinical practices.

In our work, we have already identified eye contact and gaze aversion in clinical interactions as indicative of symptoms of psychotic disorders. Our experiments also show relationships between psychotic symptoms and acoustic descriptors related to voice quality consistency, a variation of speech rate and volume, vowel space, and a parameter of glottal flow. Further, we show the link between facial expressivity and smiling behavior, as well as the connection between facial expressivity and particular psychosis symptoms.