EUVIMEDEUVIMED entdecken
Crossref

NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads

Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, Matthias Nießner

ACM Transactions on Graphics · 2023-08 · 42 (4) · 1-14

DOI: 10.1145/3592455 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 26.07.2023. Verfasst von Tobias Kirschstein und 4 weiteren Autor:innen. Die Arbeit wurde bisher 129-mal zitiert (Crossref) und nennt 57 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We focus on reconstructing high-fidelity radiance fields of human heads, capturing their animations over time, and synthesizing re-renderings from novel viewpoints at arbitrary time steps. To this end, we propose a new multi-view capture setup composed of 16 calibrated machine vision cameras that record time-synchronized images at 7.1 MP resolution and 73 frames per second. With our setup, we collect a new dataset of over 4700 high-resolution, high-framerate sequences of more than 220 human heads, from which we introduce a new human head reconstruction benchmark 1 . The recorded sequences cover a wide range of facial dynamics, including head motions, natural expressions, emotions, and spoken language. In order to reconstruct high-fidelity human heads, we propose Dynamic Neural Radiance Fields using Hash Ensembles (NeRSemble). We represent scene dynamics by combining a deformation field and an ensemble of 3D multi-resolution hash encodings. The deformation field allows for precise modeling of simple scene movements, while the ensemble of hash encodings helps to represent complex dynamics. As a result, we obtain radiance field representations of human heads that capture motion over time and facilitate re-rendering of arbitrary novel viewpoints. In a series of experiments, we explore the design choices of our method and demonstrate that our approach outperforms state-of-the-art dynamic radiance field approaches by a significant margin.

Zitieren

APA

Kirschstein, T., Qian, S., Giebenhain, S., Walter, T., & Nießner, M. (2023). NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads. ACM Transactions on Graphics, 42(4), 1-14. https://doi.org/10.1145/3592455

Vancouver

Kirschstein T, Qian S, Giebenhain S, Walter T, Nießner M. NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads. ACM Transactions on Graphics. 2023;42(4):1-14. doi:10.1145/3592455

BibTeX
@article{Kirschstein2023,
  author = {Kirschstein, Tobias and Qian, Shenhan and Giebenhain, Simon and Walter, Tim and Nießner, Matthias},
  title = {NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads},
  journal = {ACM Transactions on Graphics},
  year = {2023},
  volume = {42},
  number = {4},
  pages = {1--14},
  doi = {10.1145/3592455}
}
Kostenlos starten →