EUVIMEDEUVIMED entdecken
Crossref

THGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splatting

Chuang Chen, Lingyun Yu, Quanwei Yang, Aihua Zheng, Hongtao Xie

Computer Graphics Forum · 2025-02 · 44 (1)

DOI: 10.1111/cgf.15282 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computer Graphics Forum, online erschienen am 25.01.2025. Verfasst von Chuang Chen und 4 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 54 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite the remarkable progress in 3D talking head generation, directly generating 3D talking human avatars still suffers from rigid facial expressions, distorted hand textures and out‐of‐sync lip movements. In this paper, we extend speaker‐specific talking head generation task to talking human avatar synthesis and propose a novel pipeline, THGS , that animates lifelike Talking Human avatars using 3D Gaussian Splatting (3DGS). Given speech audio, expression and body poses as input, THGS effectively overcomes the limitations of 3DGS human re‐construction methods in capturing expressive dynamics, such as mouth movements, facial expressions and hand gestures , from a short monocular video. Firstly, we introduce a simple yet effective Learnable Expression Blendshapes (LEB) for facial dynamics re‐construction, where subtle facial dynamics can be generated by linearly combining the static head model and expression blendshapes. Secondly, a Spatial Audio Attention Module (SAAM) is proposed for lip‐synced mouth movement animation, building connections between speech audio and mouth Gaussian movements. Thirdly, we employ a body pose, expression and skinning weights joint optimization strategy to optimize these parameters on the fly, which aligns hand movements and expressions better with video input. Experimental results demonstrate that THGS can achieve high‐fidelity 3D talking human avatar animation at 150+ fps on a web‐based rendering system, improving the requirements of real‐time applications. Our project page is at https://sora158.github.io/THGS.github.io/ .

Zitieren

APA

Chen, C., Yu, L., Yang, Q., Zheng, A., & Xie, H. (2025). THGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splatting. Computer Graphics Forum, 44(1). https://doi.org/10.1111/cgf.15282

Vancouver

Chen C, Yu L, Yang Q, Zheng A, Xie H. THGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splatting. Computer Graphics Forum. 2025;44(1). doi:10.1111/cgf.15282

BibTeX
@article{Chen2025,
  author = {Chen, Chuang and Yu, Lingyun and Yang, Quanwei and Zheng, Aihua and Xie, Hongtao},
  title = {THGS: Lifelike Talking Human Avatar Synthesis From Monocular Video Via 3D Gaussian Splatting},
  journal = {Computer Graphics Forum},
  year = {2025},
  volume = {44},
  number = {1},
  doi = {10.1111/cgf.15282}
}
Kostenlos starten →