PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention
Sensors · 25.03.2023 · 23 (7) · 3447
DOI: 10.3390/s23073447 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 25.03.2023. Verfasst von Nikolas Ebert und 2 weiteren Autor:innen. Beteiligte Einrichtung: Research and Transfer Center CeMOS, Mannheim University of Applied Sciences, 68163 Mannheim, Germany. Die Arbeit wurde bisher 36-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Albert and Anneliese Konanz Foundation,, Deutsche Forschungsgemeinschaft, Federal Ministry of Education and Research.
Abstract (Originalsprache)
Recently, transformer architectures have shown superior performance compared to their CNN counterparts in many computer vision tasks. The self-attention mechanism enables transformer networks to connect visual dependencies over short as well as long distances, thus generating a large, sometimes even a global receptive field. In this paper, we propose our Parallel Local-Global Vision Transformer (PLG-ViT), a general backbone model that fuses local window self-attention with global self-attention. By merging these local and global features, short- and long-range spatial interactions can be effectively and efficiently represented without the need for costly computational operations such as shifted windows. In a comprehensive evaluation, we demonstrate that our PLG-ViT outperforms CNN-based as well as state-of-the-art transformer-based architectures in image classification and in complex downstream tasks such as object detection, instance segmentation, and semantic segmentation. In particular, our PLG-ViT models outperformed similarly sized networks like ConvNeXt and Swin Transformer, achieving Top-1 accuracy values of 83.4%, 84.0%, and 84.5% on ImageNet-1K with 27M, 52M, and 91M parameters, respectively.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Research and Transfer Center CeMOS, Mannheim University of Applied Sciences, 68163 Mannheim, Germany.
- Department of Computer Science, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany.
ORCID-Profile: Nikolas Ebert, Didier Stricker
Förderung
- Albert and Anneliese Konanz Foundation,
- Deutsche Forschungsgemeinschaft
- Federal Ministry of Education and Research
- Albert and Anneliese Konanz Foundation
- the German Research Foundation
- Research Germany in the project M2Aind-DeepLearning
Zitieren
APA
Ebert, N., Stricker, D., & Wasenmüller, O. (2023). PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention. Sensors, 23(7), 3447. https://doi.org/10.3390/s23073447
Vancouver
Ebert N, Stricker D, Wasenmüller O. PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention. Sensors. 2023;23(7):3447. doi:10.3390/s23073447
BibTeX
@article{Ebert2023,
author = {Ebert, Nikolas and Stricker, Didier and Wasenmüller, Oliver},
title = {PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention},
journal = {Sensors},
year = {2023},
volume = {23},
number = {7},
pages = {3447},
doi = {10.3390/s23073447}
}
EUVIMED