HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion
ACM Transactions on Graphics · 2023-08 · 42 (4) · 1-12
DOI: 10.1145/3592415 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 26.07.2023. Verfasst von Mustafa Işık und 6 weiteren Autor:innen. Die Arbeit wurde bisher 169-mal zitiert (Crossref) und nennt 61 Literaturangaben.
Abstract (Originalsprache)
Representing human performance at high-fidelity is an essential building block in diverse applications, such as film production, computer games or videoconferencing. To close the gap to production-level quality, we introduce HumanRF 1 , a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input, and enables playback from novel, unseen viewpoints. Our novel representation acts as a dynamic video encoding that captures fine details at high compression rates by factorizing space-time into a temporal matrix-vector decomposition. This allows us to obtain temporally coherent reconstructions of human actors for long sequences, while representing high-resolution details even in the context of challenging motion. While most research focuses on synthesizing at resolutions of 4MP or lower, we address the challenge of operating at 12MP. To this end, we introduce ActorsHQ, a novel multi-view dataset that provides 12MP footage from 160 cameras for 16 sequences with high-fidelity, per-frame mesh reconstructions 2 . We demonstrate challenges that emerge from using such high-resolution data and show that our newly introduced HumanRF effectively leverages this data, making a significant step towards production-level quality novel view synthesis.
Zitieren
APA
Işık, M., Rünz, M., Georgopoulos, M., Khakhulin, T., Starck, J., Agapito, L., & Nießner, M. (2023). HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion. ACM Transactions on Graphics, 42(4), 1-12. https://doi.org/10.1145/3592415
Vancouver
Işık M, Rünz M, Georgopoulos M, Khakhulin T, Starck J, Agapito L, et al. HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion. ACM Transactions on Graphics. 2023;42(4):1-12. doi:10.1145/3592415
BibTeX
@article{Ik2023,
author = {Işık, Mustafa and Rünz, Martin and Georgopoulos, Markos and Khakhulin, Taras and Starck, Jonathan and Agapito, Lourdes and Nießner, Matthias},
title = {HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion},
journal = {ACM Transactions on Graphics},
year = {2023},
volume = {42},
number = {4},
pages = {1--12},
doi = {10.1145/3592415}
}
EUVIMED