EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Toward Generalized Emotion Recognition in VR by Bridging Natural and Acted Facial Expressions

Rahat Rizvi Rahman, Hee Yun Choi, Joonghyo Lim, Go Eun Lee, Seungmoo Lee, Chungyean Cho, Kostadin Damevski iD

Sensors · 28.01.2026 · 26 (3) · 845

DOI: 10.3390/s26030845 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 28.01.2026. Verfasst von Rahat Rizvi Rahman und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, College of Engineering, Virginia Commonwealth University, Richmond, VA 23220, USA. Hauptthemen laut MEDLINE: Facial Expression, Emotions, Virtual Reality. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Korea Creative Content Agency.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recognizing emotions accurately in virtual reality (VR) enables adaptive and personalized experiences across gaming, therapy, and other domains. However, most existing facial emotion recognition models rely on acted expressions collected under controlled settings, which differ substantially from the spontaneous and subtle emotions that arise during real VR experiences. To address this challenge, the objective of this study is to develop and evaluate generalizable emotion recognition models that jointly learn from both acted and natural facial expressions in virtual reality. We integrate two complementary datasets collected using the Meta Quest Pro headset, one capturing natural emotional reactions and another containing acted expressions. We evaluate multiple model architectures, including convolutional and domain-adversarial networks, and a mixture-of-experts model that separates natural and acted expressions. Our experiments show that models trained jointly on acted and natural data achieve stronger cross-domain generalization. In particular, the domain-adversarial and mixture-of-experts configurations yield the highest accuracy on natural and mixed-emotion evaluations. Analysis of facial action units (AUs) reveals that natural and acted emotions rely on partially distinct AU patterns, while generalizable models learn a shared representation that integrates salient AUs from both domains. These findings demonstrate that bridging acted and natural expression domains can enable more accurate and robust VR emotion recognition systems.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Facial ExpressionEmotionsVirtual RealityHumans

Stichwörter der Autor:innen

domain adaptive neural networksfacial action coding systemhuman emotion recognitionmixture-of-expertsvirtual reality

Autor:innen und Institutionen

  • Department of Computer Science, College of Engineering, Virginia Commonwealth University, Richmond, VA 23220, USA.
  • School of Film, TV & Multimedia, Korea National University of Arts, Seoul 02789, Republic of Korea.

ORCID-Profile: Kostadin Damevski

Förderung

  • Korea Creative Content Agency

Zitieren

APA

Rahman, R. R., Choi, H. Y., Lim, J., Lee, G. E., Lee, S., Cho, C., & Damevski, K. (2026). Toward Generalized Emotion Recognition in VR by Bridging Natural and Acted Facial Expressions. Sensors, 26(3), 845. https://doi.org/10.3390/s26030845

Vancouver

Rahman RR, Choi HY, Lim J, Lee GE, Lee S, Cho C, et al. Toward Generalized Emotion Recognition in VR by Bridging Natural and Acted Facial Expressions. Sensors. 2026;26(3):845. doi:10.3390/s26030845

BibTeX
@article{Rahman2026,
  author = {Rahman, Rahat Rizvi and Choi, Hee Yun and Lim, Joonghyo and Lee, Go Eun and Lee, Seungmoo and Cho, Chungyean and Damevski, Kostadin},
  title = {Toward Generalized Emotion Recognition in VR by Bridging Natural and Acted Facial Expressions},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {3},
  pages = {845},
  doi = {10.3390/s26030845}
}
Kostenlos starten →