EUVIMEDEUVIMED entdecken
Crossref

CrossFormer++: A Versatile Vision Transformer Hinging on Cross-Scale Attention

Wenxiao Wang, Wei Chen, Qibo Qiu, Long Chen, Boxi Wu, Binbin Lin, Xiaofei He, Wei Liu

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-05 · 46 (5) · 3123-3136

DOI: 10.1109/tpami.2023.3341806 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-05. Verfasst von Wenxiao Wang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 276-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Key R&D Program of Zhejiang Province, China, Yongjiang Talent Introduction Programme.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

While features of different scales are perceptually important to visual inputs, existing vision transformers do not yet take advantage of them explicitly. To this end, we first propose a cross-scale vision transformer, CrossFormer. It introduces a cross-scale embedding layer (CEL) and a long-short distance attention (LSDA). On the one hand, CEL blends each token with multiple patches of different scales, providing the self-attention module itself with cross-scale features. On the other hand, LSDA splits the self-attention module into a short-distance one and a long-distance counterpart, which not only reduces the computational burden but also keeps both small-scale and large-scale features in the tokens. Moreover, through experiments on CrossFormer, we observe another two issues that affect vision transformers' performance, i.e., the enlarging self-attention maps and amplitude explosion. Thus, we further propose a progressive group size (PGS) paradigm and an amplitude cooling layer (ACL) to alleviate the two issues, respectively. The CrossFormer incorporating with PGS and ACL is called CrossFormer++. Extensive experiments show that CrossFormer++ outperforms the other vision transformers on image classification, object detection, instance segmentation, and semantic segmentation tasks.

Förderung

  • National Natural Science Foundation of China
  • Key R&D Program of Zhejiang Province, China
  • Yongjiang Talent Introduction Programme

Zitieren

APA

Wang, W., Chen, W., Qiu, Q., Chen, L., Wu, B., Lin, B., He, X., & Liu, W. (2024). CrossFormer++: A Versatile Vision Transformer Hinging on Cross-Scale Attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5), 3123-3136. https://doi.org/10.1109/tpami.2023.3341806

Vancouver

Wang W, Chen W, Qiu Q, Chen L, Wu B, Lin B, et al. CrossFormer++: A Versatile Vision Transformer Hinging on Cross-Scale Attention. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(5):3123-3136. doi:10.1109/tpami.2023.3341806

BibTeX
@article{Wang2024,
  author = {Wang, Wenxiao and Chen, Wei and Qiu, Qibo and Chen, Long and Wu, Boxi and Lin, Binbin and He, Xiaofei and Liu, Wei},
  title = {CrossFormer++: A Versatile Vision Transformer Hinging on Cross-Scale Attention},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {5},
  pages = {3123--3136},
  doi = {10.1109/tpami.2023.3341806}
}
Kostenlos starten →