EUVIMEDEUVIMED entdecken
Crossref

ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment

Yicheng Zhong, Huawei Wei, Peiji Yang, Zhisheng Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (7) · 7614-7622

DOI: 10.1609/aaai.v38i7.28594 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yicheng Zhong und 3 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The objective of stylized speech-driven facial animation is to create animations that encapsulate specific emotional expressions. Existing methods often depend on pre-established emotional labels or facial expression templates, which may limit the necessary flexibility for accurately conveying user intent. In this research, we introduce a technique that enables the control of arbitrary styles by leveraging natural language as emotion prompts. This technique presents benefits in terms of both flexibility and user-friendliness. To realize this objective, we initially construct a Text-Expression Alignment Dataset (TEAD), wherein each facial expression is paired with several prompt-like descriptions. We propose an innovative automatic annotation method, supported by CahtGPT, to expedite the dataset construction, thereby eliminating the substantial expense of manual annotation. Following this, we utilize TEAD to train a CLIP-based model, termed ExpCLIP, which encodes text and facial expressions into semantically aligned style embeddings. The embeddings are subsequently integrated into the facial animation generator to yield expressive and controllable facial animations. Given the limited diversity of facial emotions in existing speech-driven facial animation training data, we further introduce an effective Expression Prompt Augmentation (EPA) mechanism to enable the animation generator to support unprecedented richness in style control. Comprehensive experiments illustrate that our method accomplishes expressive facial animation generation and offers enhanced flexibility in effectively conveying the desired style.

Zitieren

APA

Zhong, Y., Wei, H., Yang, P., & Wang, Z. (2024). ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 38(7), 7614-7622. https://doi.org/10.1609/aaai.v38i7.28594

Vancouver

Zhong Y, Wei H, Yang P, Wang Z. ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(7):7614-7622. doi:10.1609/aaai.v38i7.28594

BibTeX
@article{Zhong2024,
  author = {Zhong, Yicheng and Wei, Huawei and Yang, Peiji and Wang, Zhisheng},
  title = {ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {7},
  pages = {7614--7622},
  doi = {10.1609/aaai.v38i7.28594}
}
Kostenlos starten →