EUVIMEDEUVIMED entdecken
Crossref

Unifying Flow, Stereo and Depth Estimation

Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, Andreas Geiger

IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.11.2023 · 45 (11) · 13941-13958

DOI: 10.1109/tpami.2023.3298645 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.11.2023. Verfasst von Haofei Xu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 289-mal zitiert (Crossref) und nennt 103 Literaturangaben. Gefördert von: ARC, ERC Starting, DFG EXC.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present a unified formulation and model for three motion and 3D perception tasks: optical flow, rectified stereo matching and unrectified stereo depth estimation from posed images. Unlike previous specialized architectures for each specific task, we formulate all three tasks as a unified dense correspondence matching problem, which can be solved with a single model by directly comparing feature similarities. Such a formulation calls for discriminative feature representations, which we achieve using a Transformer, in particular the cross-attention mechanism. We demonstrate that cross-attention enables integration of knowledge from another image via cross-view interactions, which greatly improves the quality of the extracted features. Our unified model naturally enables cross-task transfer since the model architecture and parameters are shared across tasks. We outperform RAFT with our unified model on the challenging Sintel dataset, and our final model that uses a few additional task-specific refinement steps outperforms or compares favorably to recent state-of-the-art methods on 10 popular flow, stereo and depth datasets, while being simpler and more efficient in terms of model design and inference speed.

Förderung

  • ARC
  • ERC Starting
  • DFG EXC

Zitieren

APA

Xu, H., Zhang, J., Cai, J., Rezatofighi, H., Yu, F., Tao, D., & Geiger, A. (2023). Unifying Flow, Stereo and Depth Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11), 13941-13958. https://doi.org/10.1109/tpami.2023.3298645

Vancouver

Xu H, Zhang J, Cai J, Rezatofighi H, Yu F, Tao D, et al. Unifying Flow, Stereo and Depth Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(11):13941-13958. doi:10.1109/tpami.2023.3298645

BibTeX
@article{Xu2023,
  author = {Xu, Haofei and Zhang, Jing and Cai, Jianfei and Rezatofighi, Hamid and Yu, Fisher and Tao, Dacheng and Geiger, Andreas},
  title = {Unifying Flow, Stereo and Depth Estimation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {11},
  pages = {13941--13958},
  doi = {10.1109/tpami.2023.3298645}
}
Kostenlos starten →