PNeRFLoc: Visual Localization with Point-Based Neural Radiance Fields
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (7) · 7450-7459
DOI: 10.1609/aaai.v38i7.28576 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Boming Zhao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref).
Abstract (Originalsprache)
Due to the ability to synthesize high-quality novel views, Neural Radiance Fields (NeRF) has been recently exploited to improve visual localization in a known environment. However, the existing methods mostly utilize NeRF for data augmentation to improve the regression model training, and their performances on novel viewpoints and appearances are still limited due to the lack of geometric constraints. In this paper, we propose a novel visual localization framework, i.e., PNeRFLoc, based on a unified point-based representation. On one hand, PNeRFLoc supports the initial pose estimation by matching 2D and 3D feature points as traditional structure-based methods; on the other hand, it also enables pose refinement with novel view synthesis using rendering-based optimization. Specifically, we propose a novel feature adaption module to close the gaps between the features for visual localization and neural rendering. To improve the efficacy and efficiency of neural rendering-based optimization, we also developed an efficient rendering-based framework with a warping loss function. Extensive experiments demonstrate that PNeRFLoc performs the best on the synthetic dataset when the 3D NeRF model can be well learned, and significantly outperforms all the NeRF-boosted localization methods with on-par SOTA performance on the real-world benchmark localization datasets. Project webpage: https://zju3dv.github.io/PNeRFLoc/.
Zitieren
APA
Zhao, B., Yang, L., Mao, M., Bao, H., & Cui, Z. (2024). PNeRFLoc: Visual Localization with Point-Based Neural Radiance Fields. Proceedings of the AAAI Conference on Artificial Intelligence, 38(7), 7450-7459. https://doi.org/10.1609/aaai.v38i7.28576
Vancouver
Zhao B, Yang L, Mao M, Bao H, Cui Z. PNeRFLoc: Visual Localization with Point-Based Neural Radiance Fields. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(7):7450-7459. doi:10.1609/aaai.v38i7.28576
BibTeX
@article{Zhao2024,
author = {Zhao, Boming and Yang, Luwei and Mao, Mao and Bao, Hujun and Cui, Zhaopeng},
title = {PNeRFLoc: Visual Localization with Point-Based Neural Radiance Fields},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {7},
pages = {7450--7459},
doi = {10.1609/aaai.v38i7.28576}
}
EUVIMED