page-people

People

Faculty

Louis-Philippe Morency

Louis-Philippe Morency

Louis-Philippe Morency is Assistant Professor in the Language Technology Institute at Carnegie Mellon University where he leads the Multimodal Communication and Machine Learning Laboratory (MultiComp Lab). He was formely research assistant professor in the Computer Sciences Department at University of Southern California and research scientist at USC Institute for Creative Technologies. Prof. Morency received his Ph.D. and Master degrees from MIT Computer Science and Artificial Intelligence Laboratory. His research focuses on building the computational foundations to enable computers with the abilities to analyze, recognize and predict subtle human communicative behaviors during social interactions. In particular, Prof. Morency was the lead investigator for the multi-institution effort that created SimSensei and MultiSense, two technologies to automatically assess nonverbal behavior indicators of psychological distress. He is currently chair of the advisory committee for ACM International Conference on Multimodal Interaction and associate editor at IEEE Transactions on Affective Computing.

Postdocs

Ph.D Students

Alex Wilf
Alex Wilf

Alex Wilf

Alex Wilf is a doctoral student in the Language Technologies Institute in the School of Computer Science. He is interested in multimodal representation learning, specifically for tasks involving how people express themselves – both when they are alone and in groups, and across both modalities and languages. He is currently interested in the promise of self-supervised learning and graph neural network architectures for use in designing novel multimodal networks. He completed his B.S. in Computer Science at the University of Michigan, where he worked with Emily Mower Provost on building robust and generalizable models for speech emotion recognition.

Martin Q. Ma
Martin Q. Ma

Martin Q. Ma

Martin Q. Ma is a Ph.D. student at the Language Technologies Institute in the School of Computer Science at Carnegie Mellon University. His research interests include deep representation learning, self-supervised learning, contrastive learning, and applications to problems in multimodal learning. Martin obtained his Master’s degree in the School of Computer Science, Carnegie Mellon University, and his Bachelor’s degree in math and computer science from Brandeis University.

Paul Pu Liang
Paul Pu Liang

Paul Pu Liang

Paul is Ph.D. student in the Machine Learning Department at CMU, advised by Louis-Philippe Morency and Ruslan Salakhutdinov. His long-term research goal is to build socially intelligent embodied agents with the ability to perceive and engage in multimodal human communication. As steps towards this goal, his research focuses on 1) the fundamentals of multimodal learning, specifically the representation, translation, fusion, and alignment of heterogeneous data sources, 2) human-centered language, vision, speech, robotics, and healthcare applications, as well as 3) the real-world deployment of socially intelligent agents by improving fairness, robustness, and interpretability in real-world applications. Previously, he received an M.S. in Machine Learning and a B.S. with University Honors in Computer Science from CMU

Victoria Lin
Victoria Lin

Victoria Lin

Victoria is a Ph.D. student in the Department of Statistics at Carnegie Mellon University. Her research interests include multimodal behavior analysis, uncertainty quantification, causal inference for high-dimensional and complex longitudinal data, and applications in mental health and medical diagnostics. Prior to joining CMU, Victoria was a researcher with Miguel Hernán in the Program for Causal Inference at the Harvard School of Public Health. She received her M.S. in data science from CMU and her joint A.B. in statistics and in molecular and cellular biology from Harvard University.

Torsten Wörtwein
Torsten Wörtwein

Torsten Wörtwein

Torsten Wörtwein is a Ph.D. student in the Language Technologies Institute. His research interests include affective computing, integrating statistical methods in deep learning, multimodal behavior analysis, and healthcare analytics with a focus on symptoms of depression and psychotic disorders. He is currently focusing on emotion recognition and its technical challenges including uncertainty estimation and personalization. Previously, he worked on public speaking anxiety and public speaking performance assessment as well as on several computer vision projects. He received his B.Sc. and M.Sc. in Informatics from the Karlsruhe Institute of Technology in Germany and a M.Sc. in Language Technologies from Carnegie Mellon University.

Alexandria Vail
Alexandria Vail

Alexandria Vail

Alexandria K. Vail is a Ph.D. student in Human-Computer Interaction. Her research interests include user modeling, affective computing, and multimodal behavior analysis, particularly within the context of healthcare and clinical decision support technologies. Her work recently received the Best Paper Award at the International Conference on User Modelling, Adaptation, and Personalization; this work has also been recognized with distinction at the International Conference on Intelligent Tutoring Systems and the International Conference on Educational Data Mining. Before joining CMU, she received the B.S. degree in Computer Science and the B.S. degree in Mathematics from North Carolina State University, with minor concentrations in Cognitive Science and Physics.

Master Students

Yiwei Lyu
Yiwei Lyu

Yiwei Lyu

Yiwei Lyu is a Master’s student in Machine Learning. His research focuses on multimodal machine learning, evaluation and interpretation of multimodal models, and multimodal pretraining/transfer learning. He has previously worked on text style transfer and automated software repair. He has earned a B.S. in Computer Science at Carnegie Mellon University.

Santiago Benoit
Santiago Benoit

Santiago Benoit

Santiago Benoit is an undergraduate student majoring in Artificial Intelligence at Carnegie Mellon University. He is currently working on generative modeling of multimodal speech from text, encoderless stochastic variational inference, and temporal sequence modeling, especially applied to music generation. His research interests are in both theoretical and applied artificial intelligence. Some of his favorite subfields in AI include multimodal machine learning, generative modeling, and reinforcement learning.

Research Engineer

Visiting Scholar

Lab Manager

Nicki Siverling
Nicki Siverling

Nicki Siverling

Nicki is the MultiComp Lab’s Coordinator. She has 12 years of experience in managing social/behavioral research labs and is currently providing support on several grants. Before joining CMU, she served as Coordinator of the Affect Analysis Group at the University of Pittsburgh. Nicki received her B.A. from the University of Pittsburgh.

Staff

Taylor Gautreaux
Taylor Gautreaux

Taylor Gautreaux

Taylor is a research associate in the MultiComp Lab. She provides support to several projects with a particular focus on data creation, processing and management. Her current research interests lie in helping intelligent systems understand human behaviors and social interactions. She previously completed a BA degree in Mass Communication with a focus on journalism at Nicholls State University.

Graduate Research Assistants

  
Yinghuan Zhang

Yinghuan Zhang

Yinghuan is a graduate student in Master of Information Technology Strategy (MITS) program at CMU. Her research interests include cross-modal supervision, representation learning, and few-/zero-shot learning in multimodal learning. Prior to joining CMU, she worked on efficient neural architecture search at Shanghai Qi Zhi Institute for 1 year. She received a B.E in Geotechnical Engineering at Harbin Institute of Technology.

  
Sanika Natu

Sanika Natu

Sanika Natu is a Master’s student in the School of Computer Science at Carnegie Mellon University. Her research interests include multimodal learning, affective computing, computational ethics, and health/behavior analytics. In the MultiComp Lab, she is currently working on research for grounding text, image, and language in educational lecture videos. Before joining her Master’s program at CMU she received her B.S. in Information Systems at CMU and worked for a few years in industry.

  
Yuxin Xiao

Yuxin Xiao

Yuxin Xiao is a Machine Learning Master student at CMU. His long-term research goal is to safely automate decision-making in society with knowledge extracted from structured data. He is currently working on quantifying the uncertainty in pre-trained language models via empirical studies. He previously earned a B.S. in Computer Science and a B.S. in Statistics and Mathematics from UIUC.

  
Blake Martin

Blake Martin

Blake is a master’s student in Machine Learning. His primary research interests are in deep learning for computer vision and natural language processing, with a current focus on multilingual machine translation. He previously received a B.S.E. is Data Science from the University of Michigan, where his research focused on machine learning applications in human health and behavior.

  
Nihal Jain

Nihal Jain

Nihal Jain is a Master’s in Machine Learning student at CMU. His research focuses on developing interpretability tools for analysis of multimodal machine learning models. He has previously worked on applications of deep learning in information retrieval when he was interning at Adobe Research. Before joining CMU, he obtained a B.E. in Computer Science at the Birla Institute of Technology & Science (BITS), Pilani, India.

  
Pratik Joshi

Pratik Joshi

Pratik Joshi is a Master’s student in the Language Technologies Institute in the School of Computer Science at Carnegie Mellon University. His research interests revolve around multimodal learning, commonsense reasoning, and multilingual systems. He is currently working on implementing continual learning techniques for multimodal generation to accommodate domain shift. Prior to joining CMU, he worked at Google Research and Microsoft Research for 2 years on problems in natural language inference, semantic parsing, and low-resource language analysis. He graduated with a Bachelor’s in Computer Science from BITS Pilani.

  
Catherine Cheng

Catherine Cheng

Catherine Cheng is a Master’s student in Machine Learning Department. Her research interests include multimodal co-learning, representation learning, and potential alignment between language and emotion representation. She has previously worked on the robustness evaluation of multimodal models. She received a B.S. in Computer Science and a B.S. in Mathematical Science at Carnegie Mellon University.

Undergraduate Research Assistants

  
Aneesha Sampath

Aneesha Sampath

Aneesha is an undergraduate student at Carnegie Mellon University majoring in Artificial Intelligence and minoring in Language Technologies. Her research interests lie in Natural Language Processing and Multimodal Deep Learning. Her current project involves uncertainty quantification.

  
Sheryl Mathew

Sheryl Mathew

Sheryl Mathew is an undergraduate computer science student at CMU. She is interested in exploring artificial intelligence, especially natural language processing and deep learning. Currently, she is working on extracting usable features for multimodal machine learning models in the social intelligence project.

  
Shuyu Zhang

Shuyu Zhang

Shuyu Zhang is an undergraduate in the BXA program studying computer science and art. She is currently working on the CMU-MOSEAS project by organizing data, processing datasets, and teaching others to do the same. Her research interests include multimodal machine learning, multimodal language models, and ethics of machine learning.

  
Edward Garemo

Edward Garemo

Edward Garemo is an undergraduate student in SCS studying computer science. He is interested in understanding and developing the theoretical foundations and practical implementations of automated generalizable learning models.

  
Evan Zhang

Evan Zhang

Hi, I’m Evan! I’m currently a freshman in SCS and plan on concentrating in machine learning. I became very interested in both software engineering and data science after completing various projects in both fields. In my free time, I like to play sports (ping pong, frisbee, basketball), listen to music (pop for sure), and hang out with friends.

  
Arvin Wu

Arvin Wu

Arvin is an undergraduate computer science student at CMU. He is interested in Multi-Modal Machine Learning, Computer Vision, and Natural Language Processing. He is currently working on a data mining project for gesture generation research.

Interns

Previous Members