EUVIMEDEUVIMED entdecken
Crossref

Vision Transformer With Quadrangle Attention

Qiming Zhang, Jing Zhang, Yufei Xu, Dacheng Tao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-05 · 46 (5) · 3608-3624

DOI: 10.1109/tpami.2023.3347693 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-05. Verfasst von Qiming Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 79-mal zitiert (Crossref) und nennt 65 Literaturangaben. Gefördert von: Australian Research Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Window-based attention has become a popular choice in vision transformers due to its superior performance, lower computational complexity, and less memory footprint. However, the design of hand-crafted windows, which is data-agnostic, constrains the flexibility of transformers to adapt to objects of varying sizes, shapes, and orientations. To address this issue, we propose a novel quadrangle attention (QA) method that extends the window-based attention to a general quadrangle formulation. Our method employs an end-to-end learnable quadrangle regression module that predicts a transformation matrix to transform default windows into target quadrangles for token sampling and attention calculation, enabling the network to model various targets with different shapes and orientations and capture rich context information. We integrate QA into plain and hierarchical vision transformers to create a new architecture named QFormer, which offers minor code modifications and negligible extra computational cost. Extensive experiments on public benchmarks demonstrate that QFormer outperforms existing representative vision transformers on various vision tasks, including classification, object detection, semantic segmentation, and pose estimation. The code will be made publicly available at QFormer.

Förderung

  • Australian Research Council

Zitieren

APA

Zhang, Q., Zhang, J., Xu, Y., & Tao, D. (2024). Vision Transformer With Quadrangle Attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5), 3608-3624. https://doi.org/10.1109/tpami.2023.3347693

Vancouver

Zhang Q, Zhang J, Xu Y, Tao D. Vision Transformer With Quadrangle Attention. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(5):3608-3624. doi:10.1109/tpami.2023.3347693

BibTeX
@article{Zhang2024,
  author = {Zhang, Qiming and Zhang, Jing and Xu, Yufei and Tao, Dacheng},
  title = {Vision Transformer With Quadrangle Attention},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {5},
  pages = {3608--3624},
  doi = {10.1109/tpami.2023.3347693}
}
Kostenlos starten →