EUVIMEDEUVIMED entdecken
Crossref

Interactive Conversational Head Generation

Mohan Zhou, Yalong Bai, Wei Zhang, Ting Yao, Tiejun Zhao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-08 · 47 (8) · 6673-6686

DOI: 10.1109/tpami.2025.3562651 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-08. Verfasst von Mohan Zhou und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Head, Communication. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 57 Literaturangaben. Gefördert von: Fundamental Research Funds for the Central Universities, National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We introduce a new conversation head generation benchmark for synthesizing behaviors of a single interlocutor in a face-to-face conversation. The capability to automatically synthesize interlocutors which can participate in long and multi-turn conversations is vital and offer benefits for various applications, including digital humans, virtual agents, and social robots. While existing research primarily focuses on talking head generation (one-way interaction), hindering the ability to create a digital human for conversation (two-way) interaction due to the absence of listening and interaction parts. In this work, we construct two datasets to address this issue, "ViCo" for independent talking and listening head generation tasks at the sentence level, and "ViCo-X", for synthesizing interlocutors in multi-turn conversational scenarios. Based on ViCo and ViCo-X, we define three novel tasks targeting the interaction modeling during the face-to-face conversation: 1) responsive listening head generation making listeners respond actively to the speaker with non-verbal signals, 2) expressive talking head generation guiding speakers to be aware of listeners' behaviors, and 3) conversational head generation to integrate the talking/listening ability in one interlocutor. Along with the datasets, we also propose corresponding baseline solutions to the three aforementioned tasks. Experimental results show that our baseline method could generate responsive and vivid agents that can collaborate with real person to fulfil the whole conversation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

HeadCommunicationHumansAlgorithmsSpeechFemaleMale

Förderung

  • Fundamental Research Funds for the Central Universities
  • National Natural Science Foundation of China

Zitieren

APA

Zhou, M., Bai, Y., Zhang, W., Yao, T., & Zhao, T. (2025). Interactive Conversational Head Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(8), 6673-6686. https://doi.org/10.1109/tpami.2025.3562651

Vancouver

Zhou M, Bai Y, Zhang W, Yao T, Zhao T. Interactive Conversational Head Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(8):6673-6686. doi:10.1109/tpami.2025.3562651

BibTeX
@article{Zhou2025,
  author = {Zhou, Mohan and Bai, Yalong and Zhang, Wei and Yao, Ting and Zhao, Tiejun},
  title = {Interactive Conversational Head Generation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {8},
  pages = {6673--6686},
  doi = {10.1109/tpami.2025.3562651}
}
Kostenlos starten →