Audio-driven 3D Facial Animation with Multimodal Context Enhancement
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2026 · 22 (11) · 1-23
DOI: 10.1145/3856308 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.11.2026. Verfasst von Yanlei Lei und 4 weiteren Autor:innen. Die Arbeit nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Key Research and Development project of Shandong Province.
Abstract (Originalsprache)
Audio-driven 3D facial animation has emerged as a prominent research topic in computer graphics. Existing autoregressive methods are often slow and suffer from error accumulation, whereas non-autoregressive methods often struggle to establish global contextual dependencies. Moreover, current approaches remain insufficient in the cross-modal semantic alignment between audio and facial animation. To address these challenges, we propose a non-autoregressive audio-driven 3D facial animation method with multimodal context enhancement. Our method fully exploits multi-scale facial representations as well as the global contextual information within facial and audio-feature sequences, enabling deep cross-modal fusion and precise semantic alignment. Specifically, we develop a global–local encoder built with spiral convolution, which facilitates complementary learning of global basic features and local fine-grained details, thereby improving the quality of facial feature extraction. We further design a multimodal context-enhanced cross-modal modulation module. This module adopts an Adaptive Instance Normalization (AdaIN)-based cross-modal fusion strategy to modulate facial features with audio sequence context in a single step, achieving fast generation without error accumulation. In addition, the module constructs a feature region cross-attention mechanism to capture contextual dependencies within facial features, and leverages globally-aware adaptive channel weighting to model temporal contextual relations of multimodal fused features. These designs enable effective exploitation of global context and substantially improve cross-modal semantic alignment accuracy. Extensive experiments demonstrate that our method outperforms mainstream baselines in both animation quality and generation efficiency.
Förderung
- National Natural Science Foundation of China
- Key Research and Development project of Shandong Province
Zitieren
APA
Lei, Y., Chi, J., Zhao, C., Xu, M., & Zhang, C. (2026). Audio-driven 3D Facial Animation with Multimodal Context Enhancement. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(11), 1-23. https://doi.org/10.1145/3856308
Vancouver
Lei Y, Chi J, Zhao C, Xu M, Zhang C. Audio-driven 3D Facial Animation with Multimodal Context Enhancement. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(11):1-23. doi:10.1145/3856308
BibTeX
@article{Lei2026,
author = {Lei, Yanlei and Chi, Jing and Zhao, Chunlin and Xu, Minfeng and Zhang, Caiming},
title = {Audio-driven 3D Facial Animation with Multimodal Context Enhancement},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2026},
volume = {22},
number = {11},
pages = {1--23},
doi = {10.1145/3856308}
}
EUVIMED