Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models
Journal of Computer Science and Frontier Technologies · 2026-04-02 · 3 (1) · 44
DOI: 10.63313/jcsft.9059 ↗At a glance
Journal article in Journal of Computer Science and Frontier Technologies, published online on 2026-04-02. Written by Zhen Wang. 19 references. The full text is freely available (open access, CC BY 4.0).
Abstract (original language)
Speech-driven 3D facial animation aims to generate realistic lip-sync and expressive facial motions from input audio. While existing methods have achieved plausible lip movements, they often struggle to capture subtle emotional variations, leading to average-looking expressions and unnatural motions. In this paper, we propose EmoFaceDiffusion, a novel framework based on denoising diffusion probabilistic models (DDPMs) that explicitly incorporates emotion awareness. Our key innovations include: (1) a multi-modal emotion encoder that extracts continuous emotion features (valence, arousal, dominance) directly from raw speech, enabling fine-grained emotional expression synthesis; (2) a conditional diffusion model with cross-modal attention that fuses audio and emotion embeddings; (3) a temporal consistency module based on graph convolutions to ensure smooth and coherent motion sequences. Extensive experiments on BIWI and IEMOCAP datasets demonstrate that EmoFaceDiffusion achieves state-of-the-art performance in lip-sync accuracy (LMD: 2.31 vs. 2.87), emotion expressiveness (classification accuracy: 78.5% vs. 65.2%), and user preference (MOS: 4.21/5). Ablation studies validate the contribution of each component. Our work offers a significant step toward expressive and emotionally aware digital avatars.
Cite
APA
Wang, Z. (2026). Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models. Journal of Computer Science and Frontier Technologies, 3(1), 44. https://doi.org/10.63313/jcsft.9059
Vancouver
Wang Z. Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models. Journal of Computer Science and Frontier Technologies. 2026;3(1):44. doi:10.63313/jcsft.9059
BibTeX
@article{Wang2026,
author = {Wang, Zhen},
title = {Emotion-Aware Speech-Driven 3D Facial Animation with Diffusion Models},
journal = {Journal of Computer Science and Frontier Technologies},
year = {2026},
volume = {3},
number = {1},
pages = {44},
doi = {10.63313/jcsft.9059}
}
EUVIMED