EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models

Zhen Wang

Journal of Computer Science and Frontier Technologies · 02.04.2026 · 3 (1) · 44

DOI: 10.63313/jcsft.9059 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Computer Science and Frontier Technologies, online erschienen am 02.04.2026. Verfasst von Zhen Wang. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Speech-driven 3D facial animation aims to generate realistic lip-sync and expressive facial motions from input audio. While existing methods have achieved plausible lip movements, they often struggle to capture subtle emotional variations, leading to average-looking expressions and unnatural motions. In this paper, we propose EmoFaceDiffusion, a novel framework based on denoising diffusion probabilistic models (DDPMs) that explicitly incorporates emotion awareness. Our key innovations include: (1) a multi-modal emotion encoder that extracts continuous emotion features (valence, arousal, dominance) directly from raw speech, enabling fine-grained emotional expression synthesis; (2) a conditional diffusion model with cross-modal attention that fuses audio and emotion embeddings; (3) a temporal consistency module based on graph convolutions to ensure smooth and coherent motion sequences. Extensive experiments on BIWI and IEMOCAP datasets demonstrate that EmoFaceDiffusion achieves state-of-the-art performance in lip-sync accuracy (LMD: 2.31 vs. 2.87), emotion expressiveness (classification accuracy: 78.5% vs. 65.2%), and user preference (MOS: 4.21/5). Ablation studies validate the contribution of each component. Our work offers a significant step toward expressive and emotionally aware digital avatars.

Zitieren

APA

Wang, Z. (2026). Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models. Journal of Computer Science and Frontier Technologies, 3(1), 44. https://doi.org/10.63313/jcsft.9059

Vancouver

Wang Z. Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models. Journal of Computer Science and Frontier Technologies. 2026;3(1):44. doi:10.63313/jcsft.9059

BibTeX
@article{Wang2026,
  author = {Wang, Zhen},
  title = {Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models},
  journal = {Journal of Computer Science and Frontier Technologies},
  year = {2026},
  volume = {3},
  number = {1},
  pages = {44},
  doi = {10.63313/jcsft.9059}
}
Kostenlos starten →