Modeling Long-range Dependencies and Epipolar Geometry for Multi-view Stereo
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2023 · 19 (6) · 1-17
DOI: 10.1145/3596445 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 12.07.2023. Verfasst von Jie Zhu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 31-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, DiDi GAIA Research Cooperation Initiative, Natural Science Foundation of Tianjin.
Abstract (Originalsprache)
This article proposes a network, referred to as Multi-View Stereo TRansformer (MVSTR) for depth estimation from multi-view images. By modeling long-range dependencies and epipolar geometry, the proposed MVSTR is capable of extracting dense features with global context and 3D consistency, which are crucial for reliable matching in multi-view stereo (MVS). Specifically, to tackle the problem of the limited receptive field of existing CNN-based MVS methods, a global-context Transformer module is designed to establish intra-view long-range dependencies so that global contextual features of each view are obtained. In addition, to further enable features of each view to be 3D consistent, a 3D-consistency Transformer module with an epipolar feature sampler is built, where epipolar geometry is modeled to effectively facilitate cross-view interaction. Experimental results show that the proposed MVSTR achieves the best overall performance on the DTU dataset and demonstrates strong generalization on the Tanks & Temples benchmark dataset.
Förderung
- National Key R&D Program of China
- National Natural Science Foundation of China
- DiDi GAIA Research Cooperation Initiative, Natural Science Foundation of Tianjin
- China Postdoctoral Science Foundation
Zitieren
APA
Zhu, J., Peng, B., Li, W., Shen, H., Huang, Q., & Lei, J. (2023). Modeling Long-range Dependencies and Epipolar Geometry for Multi-view Stereo. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(6), 1-17. https://doi.org/10.1145/3596445
Vancouver
Zhu J, Peng B, Li W, Shen H, Huang Q, Lei J. Modeling Long-range Dependencies and Epipolar Geometry for Multi-view Stereo. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(6):1-17. doi:10.1145/3596445
BibTeX
@article{Zhu2023,
author = {Zhu, Jie and Peng, Bo and Li, Wanqing and Shen, Haifeng and Huang, Qingming and Lei, Jianjun},
title = {Modeling Long-range Dependencies and Epipolar Geometry for Multi-view Stereo},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2023},
volume = {19},
number = {6},
pages = {1--17},
doi = {10.1145/3596445}
}
EUVIMED