NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads
ACM Transactions on Graphics · 2023-08 · 42 (4) · 1-14
DOI: 10.1145/3592455 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 26.07.2023. Verfasst von Tobias Kirschstein und 4 weiteren Autor:innen. Die Arbeit wurde bisher 129-mal zitiert (Crossref) und nennt 57 Literaturangaben.
Abstract (Originalsprache)
We focus on reconstructing high-fidelity radiance fields of human heads, capturing their animations over time, and synthesizing re-renderings from novel viewpoints at arbitrary time steps. To this end, we propose a new multi-view capture setup composed of 16 calibrated machine vision cameras that record time-synchronized images at 7.1 MP resolution and 73 frames per second. With our setup, we collect a new dataset of over 4700 high-resolution, high-framerate sequences of more than 220 human heads, from which we introduce a new human head reconstruction benchmark 1 . The recorded sequences cover a wide range of facial dynamics, including head motions, natural expressions, emotions, and spoken language. In order to reconstruct high-fidelity human heads, we propose Dynamic Neural Radiance Fields using Hash Ensembles (NeRSemble). We represent scene dynamics by combining a deformation field and an ensemble of 3D multi-resolution hash encodings. The deformation field allows for precise modeling of simple scene movements, while the ensemble of hash encodings helps to represent complex dynamics. As a result, we obtain radiance field representations of human heads that capture motion over time and facilitate re-rendering of arbitrary novel viewpoints. In a series of experiments, we explore the design choices of our method and demonstrate that our approach outperforms state-of-the-art dynamic radiance field approaches by a significant margin.
Zitieren
APA
Kirschstein, T., Qian, S., Giebenhain, S., Walter, T., & Nießner, M. (2023). NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads. ACM Transactions on Graphics, 42(4), 1-14. https://doi.org/10.1145/3592455
Vancouver
Kirschstein T, Qian S, Giebenhain S, Walter T, Nießner M. NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads. ACM Transactions on Graphics. 2023;42(4):1-14. doi:10.1145/3592455
BibTeX
@article{Kirschstein2023,
author = {Kirschstein, Tobias and Qian, Shenhan and Giebenhain, Simon and Walter, Tim and Nießner, Matthias},
title = {NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads},
journal = {ACM Transactions on Graphics},
year = {2023},
volume = {42},
number = {4},
pages = {1--14},
doi = {10.1145/3592455}
}
EUVIMED