EUVIMEDEUVIMED entdecken
Crossref

Frequency-Assisted Local Attention in Lower Layers of Visual Transformers

Xin Zhou iD, Zeyu Jiang iD, Shihua Zhou iD, Zhaohui Ren iD, Yongchao Zhang iD, Tianzhuang Yu iD, Yulin Liu iD

International Journal of Neural Systems · 2025-04 · 35 (04)

DOI: 10.1142/s0129065725500157 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Neural Systems, online erschienen am 28.02.2025. Verfasst von Xin Zhou und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Mechanical Engineering and Automation, Northeastern University, Wenhua Road, Shen Yang, Liao Ning, P. R. China. Hauptthemen laut MEDLINE: Attention, Neural Networks, Computer, Visual Perception. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 63 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Since vision transformers excel at establishing global relationships between features, they play an important role in current vision tasks. However, the global attention mechanism restricts the capture of local features, making convolutional assistance necessary. This paper indicates that transformer-based models can attend to local information without using convolutional blocks, similar to convolutional kernels, by employing a special initialization method. Therefore, this paper proposes a novel hybrid multi-scale model called Frequency-Assisted Local Attention Transformer (FALAT). FALAT introduces a Frequency-Assisted Window-based Positional Self-Attention (FWPSA) module that limits the attention distance of query tokens, enabling the capture of local contents in the early stage. The information from value tokens in the frequency domain enhances information diversity during self-attention computation. Additionally, the traditional convolutional method is replaced with a depth-wise separable convolution to downsample in the spatial reduction attention module for long-distance contents in the later stages. Experimental results demonstrate that FALAT-S achieves 83.0% accuracy on IN-1k with an input size of [Formula: see text] using 29.9[Formula: see text]M parameters and 5.6[Formula: see text]G FLOPs. This model outperforms the Next-ViT-S by 0.9[Formula: see text]AP b /0.8[Formula: see text]AP m with Mask-R-CNN [Formula: see text] on COCO and surpasses the recent FastViT-SA36 by 3.1% mIoU with FPN on ADE20k.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AttentionNeural Networks, ComputerVisual PerceptionHumans

Stichwörter der Autor:innen

Self-attentionimage classificationlocalityobject detectionsemantic segmentationvision transformer

Autor:innen und Institutionen

  • School of Mechanical Engineering and Automation, Northeastern University, Wenhua Road, Shen Yang, Liao Ning, P. R. China.

ORCID-Profile: Xin Zhou, Zeyu Jiang, Shihua Zhou, Zhaohui Ren, Yongchao Zhang, Tianzhuang Yu, Yulin Liu

Zitieren

APA

Zhou, X., Jiang, Z., Zhou, S., Ren, Z., Zhang, Y., Yu, T., & Liu, Y. (2025). Frequency-Assisted Local Attention in Lower Layers of Visual Transformers. International Journal of Neural Systems, 35(04). https://doi.org/10.1142/s0129065725500157

Vancouver

Zhou X, Jiang Z, Zhou S, Ren Z, Zhang Y, Yu T, et al. Frequency-Assisted Local Attention in Lower Layers of Visual Transformers. International Journal of Neural Systems. 2025;35(04). doi:10.1142/s0129065725500157

BibTeX
@article{Zhou2025,
  author = {Zhou, Xin and Jiang, Zeyu and Zhou, Shihua and Ren, Zhaohui and Zhang, Yongchao and Yu, Tianzhuang and Liu, Yulin},
  title = {Frequency-Assisted Local Attention in Lower Layers of Visual Transformers},
  journal = {International Journal of Neural Systems},
  year = {2025},
  volume = {35},
  number = {04},
  doi = {10.1142/s0129065725500157}
}
Kostenlos starten →