Deep-Learning-Derived Facial Electromyogram Signatures of Emotion in Immersive Virtual Reality (bWell): Exploring the Impact of Emotional, Cognitive, and Physical Demands
Sensors · 2026
DOI: 10.3390/s26061827 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Zohreh H Meybodi und 6 weiteren Autor:innen. Beteiligte Einrichtung: Medical Devices Research Centre, National Research Council Canada, Boucherville, QC J4B 6Y4, Canada. Hauptthemen laut MEDLINE: Electromyography, Deep Learning, Emotions, Virtual Reality. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Defense Research and Development Canada.
Abstract (Originalsprache)
Emotional and workload-related states unfold dynamically during immersive virtual reality (VR) experiences, yet reliable physiological modeling in such environments remains challenging. We investigated whether multi-channel facial electromyography (fEMG), combined with spatio-temporal deep learning, can (i) accurately classify calibrated facial expressions across participants and (ii) transfer to spontaneous, task-elicited behavior in immersive VR. Twelve adults completed a calibration phase involving four intentional expressions (smile, frown, raised eyebrow, neutral), followed by VR scenes designed to elicit emotional, cognitive, physical, and dual task demands. After participant-level physiological normalization, a single shared Convolutional Neural Network–Temporal Convolutional Network (CNN–TCN) model was trained and evaluated using leave-one-participant-out (LOPO) validation. The model achieved strong cross-participant performance (Macro-F1 = 0.88 ± 0.13; ROC-AUC = 0.95 ± 0.06). When applied to unlabeled spontaneous VR task-elicited fEMG recordings, the trained model generated continuous expression classes. Derived static and temporal expression features showed scene-dependent modulation and False Discovery Rate (FDR)-surviving associations, primarily with perceived physical demand (NASA-TLX). The observed muscle activation patterns were physiologically plausible and aligned with Facial Action Coding System (FACS)-based interpretations of underlying muscle activity. These findings demonstrate that end-to-end spatio-temporal modeling of raw fEMG enables facial expression sensing in immersive VR using a single shared model following physiological normalization. The proposed framework bridges calibrated expression learning and spontaneous task-elicited behavior, supporting privacy-preserving, continuous and physiologically grounded monitoring in human-centered VR applications.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Medical Devices Research Centre, National Research Council Canada, Boucherville, QC J4B 6Y4, Canada.
- Toronto Research Centre, Defence Research & Development Canada, Toronto, ON M3K 2C9, Canada.
ORCID-Profile: Joshua A Granek, Nusrat Choudhury
Förderung
- Defense Research and Development Canada
Zitieren
APA
Meybodi, Z. H., Thibault, F., Khundrakpam, B., De Luca, G., Zhang, J., Granek, J. A., & Choudhury, N. (2026). Deep-Learning-Derived Facial Electromyogram Signatures of Emotion in Immersive Virtual Reality (bWell): Exploring the Impact of Emotional, Cognitive, and Physical Demands. Sensors. https://doi.org/10.3390/s26061827
Vancouver
Meybodi ZH, Thibault F, Khundrakpam B, De Luca G, Zhang J, Granek JA, et al. Deep-Learning-Derived Facial Electromyogram Signatures of Emotion in Immersive Virtual Reality (bWell): Exploring the Impact of Emotional, Cognitive, and Physical Demands. Sensors. 2026. doi:10.3390/s26061827
BibTeX
@article{Meybodi2026,
author = {Meybodi, Zohreh H and Thibault, Francis and Khundrakpam, Budhachandra and De Luca, Gino and Zhang, Jing and Granek, Joshua A and Choudhury, Nusrat},
title = {Deep-Learning-Derived Facial Electromyogram Signatures of Emotion in Immersive Virtual Reality (bWell): Exploring the Impact of Emotional, Cognitive, and Physical Demands},
journal = {Sensors},
year = {2026},
doi = {10.3390/s26061827}
}
EUVIMED