EUVIMEDEUVIMED entdecken
Crossref

Dual Vision Transformer

Ting Yao, Yehao Li, Yingwei Pan, Yu Wang, Xiao-Ping Zhang, Tao Mei

IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.09.2023 · 45 (9) · 10870-10882

DOI: 10.1109/tpami.2023.3268446 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.09.2023. Verfasst von Ting Yao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 165-mal zitiert (Crossref) und nennt 69 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advances have presented several strategies to mitigate the computations of self-attention mechanism with high-resolution inputs. Many of these works consider decomposing the global self-attention procedure over image patches into regional and local feature extraction procedures that each incurs a smaller computational complexity. Despite good efficiency, these approaches seldom explore the holistic interactions among all patches, and are thus difficult to fully capture the global semantics. In this paper, we propose a novel Transformer architecture that elegantly exploits the global semantics for self-attention learning, namely Dual Vision Transformer (Dual-ViT). The new architecture incorporates a critical semantic pathway that can more efficiently compress token vectors into global semantics with reduced order of complexity. Such compressed global semantics then serve as useful prior information in learning finer local pixel level details, through another constructed pixel pathway. The semantic pathway and pixel pathway are integrated together and are jointly trained, spreading the enhanced self-attention information in parallel through both of the pathways. Dual-ViT is henceforth able to capitalize on global semantics to boost self-attention learning without compromising much computational complexity. We empirically demonstrate that Dual-ViT provides superior accuracy than SOTA Transformer architectures with comparable training complexity.

Zitieren

APA

Yao, T., Li, Y., Pan, Y., Wang, Y., Zhang, X. P., & Mei, T. (2023). Dual Vision Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9), 10870-10882. https://doi.org/10.1109/tpami.2023.3268446

Vancouver

Yao T, Li Y, Pan Y, Wang Y, Zhang XP, Mei T. Dual Vision Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(9):10870-10882. doi:10.1109/tpami.2023.3268446

BibTeX
@article{Yao2023,
  author = {Yao, Ting and Li, Yehao and Pan, Yingwei and Wang, Yu and Zhang, Xiao-Ping and Mei, Tao},
  title = {Dual Vision Transformer},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {9},
  pages = {10870--10882},
  doi = {10.1109/tpami.2023.3268446}
}
Kostenlos starten →