EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention

Nikolas Ebert iD, Didier Stricker iD, Oliver Wasenmüller

Sensors · 25.03.2023 · 23 (7) · 3447

DOI: 10.3390/s23073447 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 25.03.2023. Verfasst von Nikolas Ebert und 2 weiteren Autor:innen. Beteiligte Einrichtung: Research and Transfer Center CeMOS, Mannheim University of Applied Sciences, 68163 Mannheim, Germany. Die Arbeit wurde bisher 36-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Albert and Anneliese Konanz Foundation,, Deutsche Forschungsgemeinschaft, Federal Ministry of Education and Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recently, transformer architectures have shown superior performance compared to their CNN counterparts in many computer vision tasks. The self-attention mechanism enables transformer networks to connect visual dependencies over short as well as long distances, thus generating a large, sometimes even a global receptive field. In this paper, we propose our Parallel Local-Global Vision Transformer (PLG-ViT), a general backbone model that fuses local window self-attention with global self-attention. By merging these local and global features, short- and long-range spatial interactions can be effectively and efficiently represented without the need for costly computational operations such as shifted windows. In a comprehensive evaluation, we demonstrate that our PLG-ViT outperforms CNN-based as well as state-of-the-art transformer-based architectures in image classification and in complex downstream tasks such as object detection, instance segmentation, and semantic segmentation. In particular, our PLG-ViT models outperformed similarly sized networks like ConvNeXt and Swin Transformer, achieving Top-1 accuracy values of 83.4%, 84.0%, and 84.5% on ImageNet-1K with 27M, 52M, and 91M parameters, respectively.

Schlagwörter

Stichwörter der Autor:innen

image classificationobject detectionself-attentionsemantic segmentationtransformer

Autor:innen und Institutionen

  • Research and Transfer Center CeMOS, Mannheim University of Applied Sciences, 68163 Mannheim, Germany.
  • Department of Computer Science, RPTU Kaiserslautern-Landau, 67663 Kaiserslautern, Germany.

ORCID-Profile: Nikolas Ebert, Didier Stricker

Förderung

  • Albert and Anneliese Konanz Foundation,
  • Deutsche Forschungsgemeinschaft
  • Federal Ministry of Education and Research
  • Albert and Anneliese Konanz Foundation
  • the German Research Foundation
  • Research Germany in the project M2Aind-DeepLearning

Zitieren

APA

Ebert, N., Stricker, D., & Wasenmüller, O. (2023). PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention. Sensors, 23(7), 3447. https://doi.org/10.3390/s23073447

Vancouver

Ebert N, Stricker D, Wasenmüller O. PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention. Sensors. 2023;23(7):3447. doi:10.3390/s23073447

BibTeX
@article{Ebert2023,
  author = {Ebert, Nikolas and Stricker, Didier and Wasenmüller, Oliver},
  title = {PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {7},
  pages = {3447},
  doi = {10.3390/s23073447}
}
Kostenlos starten →