EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Development of an Interactive Digital Human with Context-Sensitive Facial Expressions

Fan Yang iD, Lei Fang iD, Rui Suo, Jing Zhang iD, Mincheol Whang iD

Sensors · 2025

DOI: 10.3390/s25165117 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Fan Yang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Emotion Engineering, Sangmyung University, Seoul 03016, Republic of Korea. Hauptthemen laut MEDLINE: Facial Expression, Emotions. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: 100 Foreign Experts Plan of Hebei Province, China Hebei Provincial Department of Science and Technology.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the increasing complexity of human–computer interaction scenarios, conventional digital human facial expression systems show notable limitations in handling multi-emotion co-occurrence, dynamic expression, and semantic responsiveness. This paper proposes a digital human system framework that integrates multimodal emotion recognition and compound facial expression generation. The system establishes a complete pipeline for real-time interaction and compound emotional expression, following a sequence of “speech semantic parsing—multimodal emotion recognition—Action Unit (AU)-level 3D facial expression control.” First, a ResNet18-based model is employed for robust emotion classification using the AffectNet dataset. Then, an AU motion curve driving module is constructed on the Unreal Engine platform, where dynamic synthesis of basic emotions is achieved via a state-machine mechanism. Finally, Generative Pre-trained Transformer (GPT) is utilized for semantic analysis, generating structured emotional weight vectors that are mapped to the AU layer to enable language-driven facial responses. Experimental results demonstrate that the proposed system significantly improves facial animation quality, with naturalness increasing from 3.54 to 3.94 and semantic congruence from 3.44 to 3.80. These results validate the system’s capability to generate realistic and emotionally coherent expressions in real time. This research provides a complete technical framework and practical foundation for high-fidelity digital humans with affective interaction capabilities.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Facial ExpressionEmotionsHumansSemanticsAlgorithms

Stichwörter der Autor:innen

action units (AUs)digital humanfacial expression generationmultimodal emotion recognitionsemantics-driven

Autor:innen und Institutionen

  • Department of Emotion Engineering, Sangmyung University, Seoul 03016, Republic of Korea.
  • College of Physical Education and Health Engineering, Hebei University of Engineering, Handan 056038, China.
  • Jingjinji Spatial Intelligent Perception Collaborative Innovation Center, Hebei University of Engineering, Handan 056009, China.
  • Department of Human-Centered Artificial Intelligence, Sangmyung University, Seoul 03016, Republic of Korea.

ORCID-Profile: Fan Yang, Lei Fang, Jing Zhang, Mincheol Whang

Förderung

  • 100 Foreign Experts Plan of Hebei Province
  • China Hebei Provincial Department of Science and Technology

Zitieren

APA

Yang, F., Fang, L., Suo, R., Zhang, J., & Whang, M. (2025). Development of an Interactive Digital Human with Context-Sensitive Facial Expressions. Sensors. https://doi.org/10.3390/s25165117

Vancouver

Yang F, Fang L, Suo R, Zhang J, Whang M. Development of an Interactive Digital Human with Context-Sensitive Facial Expressions. Sensors. 2025. doi:10.3390/s25165117

BibTeX
@article{Yang2025,
  author = {Yang, Fan and Fang, Lei and Suo, Rui and Zhang, Jing and Whang, Mincheol},
  title = {Development of an Interactive Digital Human with Context-Sensitive Facial Expressions},
  journal = {Sensors},
  year = {2025},
  doi = {10.3390/s25165117}
}
Kostenlos starten →