Vision Transformer With Quadrangle Attention
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-05 · 46 (5) · 3608-3624
DOI: 10.1109/tpami.2023.3347693 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-05. Verfasst von Qiming Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 79-mal zitiert (Crossref) und nennt 65 Literaturangaben. Gefördert von: Australian Research Council.
Abstract (Originalsprache)
Window-based attention has become a popular choice in vision transformers due to its superior performance, lower computational complexity, and less memory footprint. However, the design of hand-crafted windows, which is data-agnostic, constrains the flexibility of transformers to adapt to objects of varying sizes, shapes, and orientations. To address this issue, we propose a novel quadrangle attention (QA) method that extends the window-based attention to a general quadrangle formulation. Our method employs an end-to-end learnable quadrangle regression module that predicts a transformation matrix to transform default windows into target quadrangles for token sampling and attention calculation, enabling the network to model various targets with different shapes and orientations and capture rich context information. We integrate QA into plain and hierarchical vision transformers to create a new architecture named QFormer, which offers minor code modifications and negligible extra computational cost. Extensive experiments on public benchmarks demonstrate that QFormer outperforms existing representative vision transformers on various vision tasks, including classification, object detection, semantic segmentation, and pose estimation. The code will be made publicly available at QFormer.
Förderung
- Australian Research Council
Zitieren
APA
Zhang, Q., Zhang, J., Xu, Y., & Tao, D. (2024). Vision Transformer With Quadrangle Attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5), 3608-3624. https://doi.org/10.1109/tpami.2023.3347693
Vancouver
Zhang Q, Zhang J, Xu Y, Tao D. Vision Transformer With Quadrangle Attention. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(5):3608-3624. doi:10.1109/tpami.2023.3347693
BibTeX
@article{Zhang2024,
author = {Zhang, Qiming and Zhang, Jing and Xu, Yufei and Tao, Dacheng},
title = {Vision Transformer With Quadrangle Attention},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {5},
pages = {3608--3624},
doi = {10.1109/tpami.2023.3347693}
}
EUVIMED