EUVIMEDEUVIMED entdecken
Crossref

Animatable Implicit Neural Representations for Creating Realistic Avatars From Videos

Xiaowei Zhou, Sida Peng, Zhen Xu, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Hujun Bao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-06 · 46 (6) · 4147-4159

DOI: 10.1109/tpami.2024.3355287 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-06. Verfasst von Xiaowei Zhou und 7 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 92 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Information Technology Center, Nagoya University, State Key Lab of CAD&CG.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper addresses the challenge of reconstructing an animatable human model from a multi-view video. Some recent works have proposed to decompose a non-rigidly deforming scene into a canonical neural radiance field and a set of deformation fields that map observation-space points to the canonical space, thereby enabling them to learn the dynamic scene from images. However, they represent the deformation field as translational vector field or SE(3) field, which makes the optimization highly under-constrained. Moreover, these representations cannot be explicitly controlled by input motions. Instead, we introduce blend weight fields to produce the deformation fields. Based on the skeleton-driven deformation, blend weight fields are used with 3D human skeletons to generate observation-to-canonical and canonical-to-observation correspondences. Since 3D human skeletons are more observable, they can regularize the learning of deformation fields. Moreover, the blend weight fields can be combined with input skeletal motions to generate new deformation fields to animate the human model. To improve the quality of human modeling, we further represent the human geometry as a signed distance field in the canonical space. Additionally, a neural point displacement field is introduced to enhance the capability of the blend weight field on modeling detailed human motions. Experiments show that our approach significantly outperforms recent human modeling methods.

Förderung

  • National Natural Science Foundation of China
  • Information Technology Center, Nagoya University
  • State Key Lab of CAD&CG
  • Zhejiang University

Zitieren

APA

Zhou, X., Peng, S., Xu, Z., Dong, J., Wang, Q., Zhang, S., Shuai, Q., & Bao, H. (2024). Animatable Implicit Neural Representations for Creating Realistic Avatars From Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(6), 4147-4159. https://doi.org/10.1109/tpami.2024.3355287

Vancouver

Zhou X, Peng S, Xu Z, Dong J, Wang Q, Zhang S, et al. Animatable Implicit Neural Representations for Creating Realistic Avatars From Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(6):4147-4159. doi:10.1109/tpami.2024.3355287

BibTeX
@article{Zhou2024,
  author = {Zhou, Xiaowei and Peng, Sida and Xu, Zhen and Dong, Junting and Wang, Qianqian and Zhang, Shangzhan and Shuai, Qing and Bao, Hujun},
  title = {Animatable Implicit Neural Representations for Creating Realistic Avatars From Videos},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {6},
  pages = {4147--4159},
  doi = {10.1109/tpami.2024.3355287}
}
Kostenlos starten →