An Implicit Motion Guided Latent Completion Framework for Audio-Driven Talking Head Synthesis
Textile & Leather Review · 25.04.2026 · 9 · 2607-2628
DOI: 10.31881/tlr.2026.2607 ↗Kurz gefasst
Fachartikel in der Zeitschrift Textile & Leather Review, online erschienen am 25.04.2026. Verfasst von Xiang org/--- Li und 2 weiteren Autor:innen. Die Arbeit nennt 30 Literaturangaben.
Abstract (Originalsprache)
Audio-driven talking head synthesis requires simultaneous achievement of photorealistic visual quality, accurate lip synchronization, robust identity preservation, and computational efficiency. With the rapid proliferation of digital humans, these capabilities are becoming increasingly critical for widespread applications in human-computer interaction. Methods relying on raw audio conditioning suffer from identity-averaged dynamics, while those using explicit geometric priors (2D landmarks or 3DMM coefficients) face inherent information bottlenecks that cause over-smoothing and loss of fine-grained lip details. We present IMGTalk, an Implicit Motion Guided latent completion framework that addresses these limitations through a principled two-stage decoupled design. In Stage 1, a Transformer-based Audio-to-Implicit Motion Encoder learns a cross-identity mapping without per-subject fine-tuning from HuBERT audio features to implicit expression deformation, conditioned on each subject 's neutral silent state to enable cross-identity adaptation without per-subject fine-tuning. In Stage 2, an Implicit Motion-Conditioned UNet (IMC-UNet) completes the masked mouth region entirely within the VAE latent space, driven by the predicted expression deformation via positional-encoded cross-attention. We further introduce Reference Spatial Cross-Attention, a reference injection mechanism that replaces naive channel-wise concatenation by projecting the reference latent at every U-Net layer, enabling spatially selective retrieval of lip and teeth appearance information. Extensive experiments on the HDTF and TFHP benchmarks demonstrate that IMGTalk achieves state-of-the-art performance across all evaluation metrics.
Zitieren
APA
Li, X. O., Huang, X., & Luo, X. (2026). An Implicit Motion Guided Latent Completion Framework for Audio-Driven Talking Head Synthesis. Textile & Leather Review, 9, 2607-2628. https://doi.org/10.31881/tlr.2026.2607
Vancouver
Li XO, Huang X, Luo X. An Implicit Motion Guided Latent Completion Framework for Audio-Driven Talking Head Synthesis. Textile & Leather Review. 2026;9:2607-2628. doi:10.31881/tlr.2026.2607
BibTeX
@article{Li2026,
author = {Li, Xiang org/--- and Huang, Xiaoli and Luo, Xiaonan},
title = {An Implicit Motion Guided Latent Completion Framework for Audio-Driven Talking Head Synthesis},
journal = {Textile & Leather Review},
year = {2026},
volume = {9},
pages = {2607--2628},
doi = {10.31881/tlr.2026.2607}
}
EUVIMED