EUVIMEDEUVIMED entdecken
Crossref

ConsistTalk: Intensity Controllable Temporally Consistent Talking Head Generation with Diffusion Noise Search

Zhenjie Liu, Jianzhang Lu, Renjie Lu, Cong Liang, Shangfei Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (9) · 7449-7457

DOI: 10.1609/aaai.v40i9.37684 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhenjie Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advancements in video diffusion models have significantly enhanced audio-driven portrait animation. However, current methods still suffer from flickering, identity drift, and poor audio-visual synchronization. These issues primarily stem from entangled appearance-motion representations and unstable inference strategies. In this paper, we introduce ConsistTalk, a novel intensity-controllable and temporally consistent talking head generation framework with diffusion noise search inference. First, we propose an optical flow-guided temporal module (OFT) that decouples motion features from static appearance by leveraging facial optical flow, thereby reducing visual flicker and improving temporal consistency. Second, we present an Audio-to-Intensity (A2I) model obtained through multimodal teacher-student knowledge distillation. By transforming audio and facial velocity features into a frame-wise intensity sequence, the A2I model enables joint modeling of audio and visual motion, resulting in more natural dynamics. This further enables fine-grained, frame-wise control of motion dynamics while maintaining tight audio-visual synchronization. Third, we introduce a diffusion noise initialization strategy (IC-Init). By enforcing explicit constraints on background coherence and motion continuity during inference-time noise search, we achieve better identity preservation and refine motion dynamics compared to the current autoregressive strategy. Extensive experiments demonstrate that ConsistTalk significantly outperforms prior methods in reducing flicker, preserving identity, and delivering temporally stable, high-fidelity talking head videos.

Zitieren

APA

Liu, Z., Lu, J., Lu, R., Liang, C., & Wang, S. (2026). ConsistTalk: Intensity Controllable Temporally Consistent Talking Head Generation with Diffusion Noise Search. Proceedings of the AAAI Conference on Artificial Intelligence, 40(9), 7449-7457. https://doi.org/10.1609/aaai.v40i9.37684

Vancouver

Liu Z, Lu J, Lu R, Liang C, Wang S. ConsistTalk: Intensity Controllable Temporally Consistent Talking Head Generation with Diffusion Noise Search. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(9):7449-7457. doi:10.1609/aaai.v40i9.37684

BibTeX
@article{Liu2026,
  author = {Liu, Zhenjie and Lu, Jianzhang and Lu, Renjie and Liang, Cong and Wang, Shangfei},
  title = {ConsistTalk: Intensity Controllable Temporally Consistent Talking Head Generation with Diffusion Noise Search},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {9},
  pages = {7449--7457},
  doi = {10.1609/aaai.v40i9.37684}
}
Kostenlos starten →