EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs)

Ivana Petrovska, Boris Jutzi

Remote Sensing · 28.03.2024 · 16 (7) · 1188

DOI: 10.3390/rs16071188 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing, online erschienen am 28.03.2024. Verfasst von Ivana Petrovska und 1 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: KIT-Publication Fund of the Karlsruhe Institute of Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this contribution we evaluate the 3D geometry reconstructed by Neural Radiance Fields (NeRFs) of an object’s occluded parts behind obstacles through a point cloud comparison in 3D space against traditional Multi-View Stereo (MVS), addressing the accuracy and completeness. The key challenge lies in recovering the underlying geometry, completing the occluded parts of the object and investigating if NeRFs can compete against traditional MVS for scenarios where the latter falls short. In addition, we introduce a new “obSTaclE, occLusion and visibiLity constrAints” dataset named STELLA concerning transparent and non-transparent obstacles in real-world scenarios since there is no existing dataset dedicated to this problem setting to date. Considering that the density field represents the 3D geometry of NeRFs and is solely position-dependent, we propose an effective approach for extracting the geometry in the form of a point cloud. We voxelize the whole density field and apply a 3D density-gradient based Canny edge detection filter to better represent the object’s geometric features. The qualitative and quantitative results demonstrate NeRFs’ ability to capture geometric details of the occluded parts in all scenarios, thus outperforming in completeness, as our voxel-based point cloud extraction approach achieves point coverage up to 93%. However, MVS remains a more accurate image-based 3D reconstruction method, deviating from the ground truth 2.26 mm and 3.36 mm for each obstacle scenario respectively.

Förderung

  • KIT-Publication Fund of the Karlsruhe Institute of Technology

Zitieren

APA

Petrovska, I., & Jutzi, B. (2024). Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs). Remote Sensing, 16(7), 1188. https://doi.org/10.3390/rs16071188

Vancouver

Petrovska I, Jutzi B. Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs). Remote Sensing. 2024;16(7):1188. doi:10.3390/rs16071188

BibTeX
@article{Petrovska2024,
  author = {Petrovska, Ivana and Jutzi, Boris},
  title = {Vision through Obstacles—3D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs)},
  journal = {Remote Sensing},
  year = {2024},
  volume = {16},
  number = {7},
  pages = {1188},
  doi = {10.3390/rs16071188}
}
Kostenlos starten →