VSFormer: Visual-Spatial Fusion Transformer for Correspondence Pruning
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (4) · 3369-3377
DOI: 10.1609/aaai.v38i4.28123 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Tangfei Liao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref).
Abstract (Originalsprache)
Correspondence pruning aims to find correct matches (inliers) from an initial set of putative correspondences, which is a fundamental task for many applications. The process of finding is challenging, given the varying inlier ratios between scenes/image pairs due to significant visual differences. However, the performance of the existing methods is usually limited by the problem of lacking visual cues (e.g., texture, illumination, structure) of scenes. In this paper, we propose a Visual-Spatial Fusion Transformer (VSFormer) to identify inliers and recover camera poses accurately. Firstly, we obtain highly abstract visual cues of a scene with the cross attention between local features of two-view images. Then, we model these visual cues and correspondences by a joint visual-spatial fusion module, simultaneously embedding visual cues into correspondences for pruning. Additionally, to mine the consistency of correspondences, we also design a novel module that combines the KNN-based graph and the transformer, effectively capturing both local and global contexts. Extensive experiments have demonstrated that the proposed VSFormer outperforms state-of-the-art methods on outdoor and indoor benchmarks. Our code is provided at the following repository: https://github.com/sugar-fly/VSFormer.
Zitieren
APA
Liao, T., Zhang, X., Zhao, L., Wang, T., & Xiao, G. (2024). VSFormer: Visual-Spatial Fusion Transformer for Correspondence Pruning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4), 3369-3377. https://doi.org/10.1609/aaai.v38i4.28123
Vancouver
Liao T, Zhang X, Zhao L, Wang T, Xiao G. VSFormer: Visual-Spatial Fusion Transformer for Correspondence Pruning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(4):3369-3377. doi:10.1609/aaai.v38i4.28123
BibTeX
@article{Liao2024,
author = {Liao, Tangfei and Zhang, Xiaoqin and Zhao, Li and Wang, Tao and Xiao, Guobao},
title = {VSFormer: Visual-Spatial Fusion Transformer for Correspondence Pruning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {4},
pages = {3369--3377},
doi = {10.1609/aaai.v38i4.28123}
}
EUVIMED