Visibility-Aware Multi-View Stereo by Surface Normal Weighting for Occlusion Robustness
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-09 · 47 (9) · 7526-7542
DOI: 10.1109/tpami.2025.3568447 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-09. Verfasst von Hyucksang Lee und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: Culture, Sports and Tourism R&D Program through the Korea Creative Content Agency, Ministry of Culture, Sports and Tourism in 2024, Global Talent for Generative AI Copyright Infringement and Copyright Theft.
Abstract (Originalsprache)
Recent learning-based multi-view stereo (MVS) still exhibits insufficient accuracy in large occlusion cases, such as environments with significant inter-camera distance or when capturing objects with complex shapes. This is because incorrect image features extracted from occluded areas serve as significant noise in the cost volume construction. To address this, we propose a visibility-aware MVS using surface normal weighting (SnowMVSNet) based on explicit 3D geometry. It selectively suppresses mismatched features in the cost volume construction by computing inter-view visibility. Additionally, we present a geometry-guided cost volume regularization that enhances true depth among depth hypotheses using a surface normal prior. We also propose intra-view visibility that distinguishes geometrically more visible pixels within a reference view. Using intra-view visibility, we introduce the visibility-weighted training and depth estimation methods. These methods enable the network to achieve accurate 3D point cloud reconstruction by focusing on visible regions. Based on simple inter-view and intra-view visibility computations, SnowMVSNet accomplishes substantial performance improvements relative to computational complexity, particularly in terms of occlusion robustness. To evaluate occlusion robustness, we constructed a multi-view human (MVHuman) dataset containing general human body shapes prone to self-occlusion. Extensive experiments demonstrated that SnowMVSNet significantly outperformed state-of-the-art methods in both low- and high-occlusion scenarios.
Förderung
- Culture, Sports and Tourism R&D Program through the Korea Creative Content Agency
- Ministry of Culture, Sports and Tourism in 2024
- Global Talent for Generative AI Copyright Infringement and Copyright Theft
- Yonsei Signature Research Cluster Program of 2025
Zitieren
APA
Lee, H., Lee, S., & Lee, S. (2025). Visibility-Aware Multi-View Stereo by Surface Normal Weighting for Occlusion Robustness. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(9), 7526-7542. https://doi.org/10.1109/tpami.2025.3568447
Vancouver
Lee H, Lee S, Lee S. Visibility-Aware Multi-View Stereo by Surface Normal Weighting for Occlusion Robustness. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(9):7526-7542. doi:10.1109/tpami.2025.3568447
BibTeX
@article{Lee2025,
author = {Lee, Hyucksang and Lee, Seongmin and Lee, Sanghoon},
title = {Visibility-Aware Multi-View Stereo by Surface Normal Weighting for Occlusion Robustness},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {9},
pages = {7526--7542},
doi = {10.1109/tpami.2025.3568447}
}
EUVIMED