Health Behavior Informatics
Nonverbal behaviors are critically important components of the mental health diagnosis, therapeutic encounter, and clinical decision-making. In combination with content from verbal exchanges, therapists and psychiatrists use observations of nonverbal behaviors of patients to make a range of clinical decisions, including medication management, psychotherapeutic interventions, and assessments about clinical course and severity. Nonverbal interactions are typically assessed only subjectively by clinicians. There is a crucial need, emphasized by the 2015 National Institute of Mental Health (NIMH) Strategic Plan for Research, to identify clinically-useful behavioral indicators that predict psychotherapy process and outcome variables. Taking advantages of recent advances in automatic recognition of human multimodal behaviors, the emerging field of health behavioral informatics has shown the promise of using computer science techniques to illuminate and augment traditional clinical practice.
MultiComp Lab works closely with world-renowned medical centers such as Harvard Medical School, University of Pittsburgh Medical Center (UPMC) and Yale Medical School to address the challenge of health behavior informatics. Our initial research studied patient behavior indicators related to depression and post trauma stress disorders. One of our scientific discovery in the impact of gender when studying negative facial expressions is distress disorders: men show an increase in negative expressions while the opposite trend is present for women. We studied verbal and acoustic behaviors of patients with suicidal ideation, finding behavior markers in the use of pronouns, the pronunciation of vowels and the quality of the voice. Our recent analysis of multimodal behaviors from psychosis patients showing schizophrenia symptoms introduced behavior markers for facial expressiveness and vocal prosody. Automatic detection of such behavior indicators could assist clinicians by supporting their observations and by providing a more systematic measurement and quantification of nonverbal patterns both within and across clinical sessions.