SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration
ACM Transactions on Graphics · 19.07.2024 · 43 (4) · 1-13
DOI: 10.1145/3658193 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 19.07.2024. Verfasst von Daniel Duckworth und 7 weiteren Autor:innen. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Recent techniques for real-time view synthesis have rapidly advanced in fidelity and speed, and modern methods are capable of rendering near-photorealistic scenes at interactive frame rates. At the same time, a tension has arisen between explicit scene representations amenable to rasterization and neural fields built on ray marching, with state-of-the-art instances of the latter surpassing the former in quality while being prohibitively expensive for real-time applications. We introduce SMERF, a view synthesis approach that achieves state-of-the-art accuracy among real-time methods on large scenes with footprints up to 300 m 2 at a volumetric resolution of 3.5 mm 3 . Our method is built upon two primary contributions: a hierarchical model partitioning scheme, which increases model capacity while constraining compute and memory consumption, and a distillation training strategy that simultaneously yields high fidelity and internal consistency. Our method enables full six degrees of freedom navigation in a web browser and renders in real-time on commodity smartphones and laptops. Extensive experiments show that our method exceeds the state-of-the-art in real-time novel view synthesis by 0.78 dB on standard benchmarks and 1.78 dB on large scenes, renders frames three orders of magnitude faster than state-of-the-art radiance field models, and achieves real-time performance across a wide variety of commodity devices, including smartphones. We encourage readers to explore these models interactively at our project website: https://smerf-3d.github.io.
Zitieren
APA
Duckworth, D., Hedman, P., Reiser, C., Zhizhin, P., Thibert, J., Lučić, M., Szeliski, R., & Barron, J. T. (2024). SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration. ACM Transactions on Graphics, 43(4), 1-13. https://doi.org/10.1145/3658193
Vancouver
Duckworth D, Hedman P, Reiser C, Zhizhin P, Thibert J, Lučić M, et al. SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration. ACM Transactions on Graphics. 2024;43(4):1-13. doi:10.1145/3658193
BibTeX
@article{Duckworth2024,
author = {Duckworth, Daniel and Hedman, Peter and Reiser, Christian and Zhizhin, Peter and Thibert, Jean-François and Lučić, Mario and Szeliski, Richard and Barron, Jonathan T.},
title = {SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration},
journal = {ACM Transactions on Graphics},
year = {2024},
volume = {43},
number = {4},
pages = {1--13},
doi = {10.1145/3658193}
}
EUVIMED