MAVAGEN: Multimodal Avatar Generation Framework for Personalized Human–Computer Interaction
Multimodal Technologies and Interaction · 18.05.2026 · 10 (5) · 55
DOI: 10.3390/mti10050055 ↗Kurz gefasst
Fachartikel in der Zeitschrift Multimodal Technologies and Interaction, online erschienen am 18.05.2026. Verfasst von Alexandr Axyonov und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Russian Science Foundation, Russian state research.
Abstract (Originalsprache)
Digital-avatar systems still provide limited control over emotionally expressive behavior in human–computer interaction, especially in Large Language Model (LLM)-based chatbots and virtual assistants with personalized visual embodiments. To address this problem, we propose Multimodal Avatar Generation (MAVAGEN), a multimodal avatar generation framework for synthesizing upper-body digital avatars with personalized appearance and controllable emotional expression. The user specifies the desired gender and age, as well as provides a short text input from which the target emotional state is inferred. MAVAGEN then retrieves an identity image from the HaGRIDv2-1M corpus and generates an avatar clip with synchronized facial expressions, hand gestures, and expressive speech. The framework uses the following six feature streams: textual features, emotion-distribution features, landmark-based pose features, depth-geometry features, RGB-appearance features, and acoustic features. In a quantitative evaluation against recent human animation methods, MAVAGEN achieves the best overall avatar quality, with FID 48.20, FVD 592.00, SSIM 0.741, Sync-C 7.40, HKC 0.929, HKV 25.30, CSIM 0.563, and EmoAcc 0.88. Ablation results show that emotion and acoustic features contribute most to emotional agreement, while landmark-based pose and depth features improve geometric and motion stability. These results support the practical use of MAVAGEN in personalized LLM-based assistants and other emotion-sensitive interactive systems.
Förderung
- Russian Science Foundation
- Russian state research
Zitieren
APA
Axyonov, A., Ryumina, E., Ryumin, D., & Karpov, A. (2026). MAVAGEN: Multimodal Avatar Generation Framework for Personalized Human–Computer Interaction. Multimodal Technologies and Interaction, 10(5), 55. https://doi.org/10.3390/mti10050055
Vancouver
Axyonov A, Ryumina E, Ryumin D, Karpov A. MAVAGEN: Multimodal Avatar Generation Framework for Personalized Human–Computer Interaction. Multimodal Technologies and Interaction. 2026;10(5):55. doi:10.3390/mti10050055
BibTeX
@article{Axyonov2026,
author = {Axyonov, Alexandr and Ryumina, Elena and Ryumin, Dmitry and Karpov, Alexey},
title = {MAVAGEN: Multimodal Avatar Generation Framework for Personalized Human–Computer Interaction},
journal = {Multimodal Technologies and Interaction},
year = {2026},
volume = {10},
number = {5},
pages = {55},
doi = {10.3390/mti10050055}
}
EUVIMED