Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models
Journal of Computer Science and Frontier Technologies · 02.04.2026 · 3 (1) · 44
DOI: 10.63313/jcsft.9059 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Computer Science and Frontier Technologies, online erschienen am 02.04.2026. Verfasst von Zhen Wang. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Speech-driven 3D facial animation aims to generate realistic lip-sync and expressive facial motions from input audio. While existing methods have achieved plausible lip movements, they often struggle to capture subtle emotional variations, leading to average-looking expressions and unnatural motions. In this paper, we propose EmoFaceDiffusion, a novel framework based on denoising diffusion probabilistic models (DDPMs) that explicitly incorporates emotion awareness. Our key innovations include: (1) a multi-modal emotion encoder that extracts continuous emotion features (valence, arousal, dominance) directly from raw speech, enabling fine-grained emotional expression synthesis; (2) a conditional diffusion model with cross-modal attention that fuses audio and emotion embeddings; (3) a temporal consistency module based on graph convolutions to ensure smooth and coherent motion sequences. Extensive experiments on BIWI and IEMOCAP datasets demonstrate that EmoFaceDiffusion achieves state-of-the-art performance in lip-sync accuracy (LMD: 2.31 vs. 2.87), emotion expressiveness (classification accuracy: 78.5% vs. 65.2%), and user preference (MOS: 4.21/5). Ablation studies validate the contribution of each component. Our work offers a significant step toward expressive and emotionally aware digital avatars.
Zitieren
APA
Wang, Z. (2026). Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models. Journal of Computer Science and Frontier Technologies, 3(1), 44. https://doi.org/10.63313/jcsft.9059
Vancouver
Wang Z. Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models. Journal of Computer Science and Frontier Technologies. 2026;3(1):44. doi:10.63313/jcsft.9059
BibTeX
@article{Wang2026,
author = {Wang, Zhen},
title = {Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models},
journal = {Journal of Computer Science and Frontier Technologies},
year = {2026},
volume = {3},
number = {1},
pages = {44},
doi = {10.63313/jcsft.9059}
}
EUVIMED