EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration

Daniel Duckworth, Peter Hedman, Christian Reiser, Peter Zhizhin, Jean-François Thibert, Mario Lučić, Richard Szeliski, Jonathan T. Barron

ACM Transactions on Graphics · 19.07.2024 · 43 (4) · 1-13

DOI: 10.1145/3658193 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 19.07.2024. Verfasst von Daniel Duckworth und 7 weiteren Autor:innen. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent techniques for real-time view synthesis have rapidly advanced in fidelity and speed, and modern methods are capable of rendering near-photorealistic scenes at interactive frame rates. At the same time, a tension has arisen between explicit scene representations amenable to rasterization and neural fields built on ray marching, with state-of-the-art instances of the latter surpassing the former in quality while being prohibitively expensive for real-time applications. We introduce SMERF, a view synthesis approach that achieves state-of-the-art accuracy among real-time methods on large scenes with footprints up to 300 m 2 at a volumetric resolution of 3.5 mm 3 . Our method is built upon two primary contributions: a hierarchical model partitioning scheme, which increases model capacity while constraining compute and memory consumption, and a distillation training strategy that simultaneously yields high fidelity and internal consistency. Our method enables full six degrees of freedom navigation in a web browser and renders in real-time on commodity smartphones and laptops. Extensive experiments show that our method exceeds the state-of-the-art in real-time novel view synthesis by 0.78 dB on standard benchmarks and 1.78 dB on large scenes, renders frames three orders of magnitude faster than state-of-the-art radiance field models, and achieves real-time performance across a wide variety of commodity devices, including smartphones. We encourage readers to explore these models interactively at our project website: https://smerf-3d.github.io.

Zitieren

APA

Duckworth, D., Hedman, P., Reiser, C., Zhizhin, P., Thibert, J., Lučić, M., Szeliski, R., & Barron, J. T. (2024). SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration. ACM Transactions on Graphics, 43(4), 1-13. https://doi.org/10.1145/3658193

Vancouver

Duckworth D, Hedman P, Reiser C, Zhizhin P, Thibert J, Lučić M, et al. SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration. ACM Transactions on Graphics. 2024;43(4):1-13. doi:10.1145/3658193

BibTeX
@article{Duckworth2024,
  author = {Duckworth, Daniel and Hedman, Peter and Reiser, Christian and Zhizhin, Peter and Thibert, Jean-François and Lučić, Mario and Szeliski, Richard and Barron, Jonathan T.},
  title = {SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration},
  journal = {ACM Transactions on Graphics},
  year = {2024},
  volume = {43},
  number = {4},
  pages = {1--13},
  doi = {10.1145/3658193}
}
Kostenlos starten →