page-people

People

Faculty

Louis-Philippe Morency

Louis-Philippe Morency

Louis-Philippe Morency is Assistant Professor in the Language Technology Institute at Carnegie Mellon University where he leads the Multimodal Communication and Machine Learning Laboratory (MultiComp Lab). He was formely research assistant professor in the Computer Sciences Department at University of Southern California and research scientist at USC Institute for Creative Technologies. Prof. Morency received his Ph.D. and Master degrees from MIT Computer Science and Artificial Intelligence Laboratory. His research focuses on building the computational foundations to enable computers with the abilities to analyze, recognize and predict subtle human communicative behaviors during social interactions. In particular, Prof. Morency was the lead investigator for the multi-institution effort that created SimSensei and MultiSense, two technologies to automatically assess nonverbal behavior indicators of psychological distress. He is currently chair of the advisory committee for ACM International Conference on Multimodal Interaction and associate editor at IEEE Transactions on Affective Computing.

Postdocs

Ph.D Students

Alex Wilf
Alex Wilf

Alex Wilf

Alex Wilf is a doctoral student in the Language Technologies Institute in the School of Computer Science. He is interested in multimodal representation learning, specifically for tasks involving how people express themselves – both when they are alone and in groups, and across both modalities and languages. He is currently interested in the promise of self-supervised learning and graph neural network architectures for use in designing novel multimodal networks. He completed his B.S. in Computer Science at the University of Michigan, where he worked with Emily Mower Provost on building robust and generalizable models for speech emotion recognition.

Chaitanya Ahuja
Chaitanya Ahuja

Chaitanya Ahuja

Chaitanya Ahuja is a doctoral student in Language Technologies Institute in the School of Computer Science at Carnegie Mellon University. His interests range in various topics in natural language, computer vision, computational music, and machine learning. He is currently working on various representation models for multimodal machine learning with Louis-Philippe Morency. Before starting with graduate school, Chaitanya completed his B.Tech at Indian Institute of Technology, Kanpur where he worked on Speech Source Separation and Spatial Audio with Rajesh Hegde.

Volkan Cirik
Volkan Cirik

Volkan Cirik

Volkan is a doctoral student at Language Technologies Institute in the School of Computer Science. His interests range on various topics in natural language, computer vision, and machine learning. Volkan finished a Master’s degree in word vectors for structured prediction at Koç AI Lab with Deniz Yuret. During his second Master’s in Carnegie Mellon, he worked on various topics such as event detection in the biomedical domain, visualization, and analysis of neural network models with Teruko Mitamura, Eduard Hovy, and Louis-Philippe Morency. Before starting his graduate studies, he completed his B.S in computer engineering at Boğaziçi University.

Paul Pu Liang
Paul Pu Liang

Paul Pu Liang

Paul is Ph.D. student in the Machine Learning Department at CMU, advised by Louis-Philippe Morency and Ruslan Salakhutdinov. His long-term research goal is to build socially intelligent embodied agents with the ability to perceive and engage in multimodal human communication. As steps towards this goal, his research focuses on 1) the fundamentals of multimodal learning, specifically the representation, translation, fusion, and alignment of heterogeneous data sources, 2) human-centered language, vision, speech, robotics, and healthcare applications, as well as 3) the real-world deployment of socially intelligent agents by improving fairness, robustness, and interpretability in real-world applications. Previously, he received an M.S. in Machine Learning and a B.S. with University Honors in Computer Science from CMU

Yao-Hung Hubert Tsai
Yao-Hung Hubert Tsai

Yao-Hung Hubert Tsai

Hubert is a Ph.D. student in the Machine Learning Department at Carnegie Mellon University under the supervision of Dr. Ruslan Salakhutdinov and Dr. Louis-Philippe Morency. His research interests lie in Deep Learning and its applications, especially in Multi-modal and Multi-view Learning and Statistical Machine Learning. He obtained his B.S. in E.E. from National Taiwan University. He is supported by Facebook Fellowship.

Victoria Lin
Victoria Lin

Victoria Lin

Victoria is a Ph.D. student in the Department of Statistics at Carnegie Mellon University. Her research interests include multimodal behavior analysis, uncertainty quantification, causal inference for high-dimensional and complex longitudinal data, and applications in mental health and medical diagnostics. Prior to joining CMU, Victoria was a researcher with Miguel Hernán in the Program for Causal Inference at the Harvard School of Public Health. She received her M.S. in data science from CMU and her joint A.B. in statistics and in molecular and cellular biology from Harvard University.

Torsten Wörtwein
Torsten Wörtwein

Torsten Wörtwein

Torsten Wörtwein is a Ph.D. student in the Language Technologies Institute. His research interests include affective computing, integrating statistical methods in deep learning, multimodal behavior analysis, and healthcare analytics with a focus on symptoms of depression and psychotic disorders. He is currently focusing on emotion recognition and its technical challenges including uncertainty estimation and personalization. Previously, he worked on public speaking anxiety and public speaking performance assessment as well as on several computer vision projects. He received his B.Sc. and M.Sc. in Informatics from the Karlsruhe Institute of Technology in Germany and a M.Sc. in Language Technologies from Carnegie Mellon University.

Alexandria Vail
Alexandria Vail

Alexandria Vail

Alexandria K. Vail is a Ph.D. student in Human-Computer Interaction. Her research interests include user modeling, affective computing, and multimodal behavior analysis, particularly within the context of healthcare and clinical decision support technologies. Her work recently received the Best Paper Award at the International Conference on User Modelling, Adaptation, and Personalization; this work has also been recognized with distinction at the International Conference on Intelligent Tutoring Systems and the International Conference on Educational Data Mining. Before joining CMU, she received the B.S. degree in Computer Science and the B.S. degree in Mathematics from North Carolina State University, with minor concentrations in Cognitive Science and Physics.

Amir Zadeh
Amir Zadeh

Amir Zadeh

Amir Zadeh is an Artificial Intelligence Ph.D student at Carnegie Mellon University. His research is focused on Multimodal Deep Learning both theory and applied. From theoretical perspective he is interested in building foundations of multimodal machine learning. From application perspective, he is keen on giving computers capability to understand human communication as a multimodal signal involving language, gestures and voice. His research spans different areas of natural language processing, computer vision and speech processing. He started his Ph.D in January 2015. Prior to that he received his Bachelors of Science from University of Tehran, ECE department, where he was a member of Advanced Robotics Laboratory.

Master Students

Yiwei Lyu
Yiwei Lyu

Yiwei Lyu

Yiwei Lyu is a Master’s student in Machine Learning. His research focuses on multimodal machine learning, evaluation and interpretation of multimodal models, and multimodal pretraining/transfer learning. He has previously worked on text style transfer and automated software repair. He has earned a B.S. in Computer Science at Carnegie Mellon University.

Dong Won Lee
Dong Won Lee

Dong Won Lee

Don is a Machine Learning Masters student in Carnegie Mellon University. He is working to build models to understand the relationship between what we see (vision) and what we hear (language) and implement these findings to interesting applications in education, human-robot-interaction, and robotics control.

Research Engineer

Visiting Scholar

Lab Manager

Nicki Siverling
Nicki Siverling

Nicki Siverling

Nicki is the MultiComp Lab’s Coordinator. She has 12 years of experience in managing social/behavioral research labs and is currently providing support on several grants. Before joining CMU, she served as Coordinator of the Affect Analysis Group at the University of Pittsburgh. Nicki received her B.A. from the University of Pittsburgh.

Staff

Graduate Research Assistants

  
Navya Yarrabelly

Navya Yarrabelly

Navya is a Masters’s student at Language Technologies Institute in the School of Computer Science. Her interests lie in representation learning for graphs, Text to SQL, topic modeling. Her current work is on language grounding in image captions. Navya received her bachelor’s degree from IIIT-Hyderabad and has worked at Microsoft Research, where she worked with Ayush Choure and Prateek Jain on relevance ranking problems for large-scale social networks. Navya works with Volkan on identifying and linking object mentions in image captions.

  
Martin Ma

Martin Ma

Martin Q. Ma is a second-year master student in the School of Computer Science. He has been working on deep learning and multimodal learning. His previous work lies in topics including multimodal learning, seq-to-seq learning, domain adaptation, zero-shot learning, interpretable models, etc. Martin received his Bachelor’s degree in computer science and mathematics from Brandeis University.

  
Jianing “Jed” Yang

Jianing “Jed” Yang

Jianing “Jed” Yang is a second-year Master’s student in Machine Learning. His research interest lies primarily in Natural Language Processing and Machine Learning. His projects include question answering bias analysis on multimodal QA datasets and multimodal fusion methods with unaligned multimodal language sequences. Before joining CMU, Jed received his Bachelor’s in Computer Science from Georgia Tech and was doing research with Prof. Jimeng Sun on Machine Learning with Healthcare.

Undergraduate Research Assistants

  
Niki Murikinati

Niki Murikinati

Niki is a junior studying Computer Science and minoring in Machine Learning at Carnegie Mellon. Her interests lie in Multimodal Machine Learning and Natural Language Processing. She is currently working on a project involving generating speech with prosodic features from visual gesture data and text transcripts.

  
Arvin Wu

Arvin Wu

Arvin is an undergraduate computer science student at CMU. He is interested in Multi-Modal Machine Learning, Computer Vision, and Natural Language Processing. He is currently working on a data mining project for gesture generation research.

  
Santiago Benoit

Santiago Benoit

Santiago Benoit is an undergraduate student majoring in Artificial Intelligence at Carnegie Mellon University. He is currently working on generative modeling of multimodal speech (video and audio) from text, long-range sequence modeling, and encoderless stochastic variational inference (with Amir Zadeh). His research interests are in both theoretical and applied research in artificial intelligence. Some of his favorite subfields in AI include multimodal machine learning, generative models, and reinforcement learning.

  
Katrina Jiao

Katrina Jiao

Katrina is an undergraduate student in Computer Science. Her research interests lie in representation learning for Reinforcement Learning tasks and multimodal machine learning techniques. Her current work concerns improving the language grounding process in text-assisted RL in order to facilitate the transfer between similar video game environments.

Interns

Previous Members