Frequency-Assisted Local Attention in Lower Layers of Visual Transformers
International Journal of Neural Systems · 2025-04 · 35 (04)
DOI: 10.1142/s0129065725500157 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Neural Systems, online erschienen am 28.02.2025. Verfasst von Xin Zhou und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Mechanical Engineering and Automation, Northeastern University, Wenhua Road, Shen Yang, Liao Ning, P. R. China. Hauptthemen laut MEDLINE: Attention, Neural Networks, Computer, Visual Perception. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 63 Literaturangaben.
Abstract (Originalsprache)
Since vision transformers excel at establishing global relationships between features, they play an important role in current vision tasks. However, the global attention mechanism restricts the capture of local features, making convolutional assistance necessary. This paper indicates that transformer-based models can attend to local information without using convolutional blocks, similar to convolutional kernels, by employing a special initialization method. Therefore, this paper proposes a novel hybrid multi-scale model called Frequency-Assisted Local Attention Transformer (FALAT). FALAT introduces a Frequency-Assisted Window-based Positional Self-Attention (FWPSA) module that limits the attention distance of query tokens, enabling the capture of local contents in the early stage. The information from value tokens in the frequency domain enhances information diversity during self-attention computation. Additionally, the traditional convolutional method is replaced with a depth-wise separable convolution to downsample in the spatial reduction attention module for long-distance contents in the later stages. Experimental results demonstrate that FALAT-S achieves 83.0% accuracy on IN-1k with an input size of [Formula: see text] using 29.9[Formula: see text]M parameters and 5.6[Formula: see text]G FLOPs. This model outperforms the Next-ViT-S by 0.9[Formula: see text]AP b /0.8[Formula: see text]AP m with Mask-R-CNN [Formula: see text] on COCO and surpasses the recent FastViT-SA36 by 3.1% mIoU with FPN on ADE20k.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Mechanical Engineering and Automation, Northeastern University, Wenhua Road, Shen Yang, Liao Ning, P. R. China.
ORCID-Profile: Xin Zhou, Zeyu Jiang, Shihua Zhou, Zhaohui Ren, Yongchao Zhang, Tianzhuang Yu, Yulin Liu
Zitieren
APA
Zhou, X., Jiang, Z., Zhou, S., Ren, Z., Zhang, Y., Yu, T., & Liu, Y. (2025). Frequency-Assisted Local Attention in Lower Layers of Visual Transformers. International Journal of Neural Systems, 35(04). https://doi.org/10.1142/s0129065725500157
Vancouver
Zhou X, Jiang Z, Zhou S, Ren Z, Zhang Y, Yu T, et al. Frequency-Assisted Local Attention in Lower Layers of Visual Transformers. International Journal of Neural Systems. 2025;35(04). doi:10.1142/s0129065725500157
BibTeX
@article{Zhou2025,
author = {Zhou, Xin and Jiang, Zeyu and Zhou, Shihua and Ren, Zhaohui and Zhang, Yongchao and Yu, Tianzhuang and Liu, Yulin},
title = {Frequency-Assisted Local Attention in Lower Layers of Visual Transformers},
journal = {International Journal of Neural Systems},
year = {2025},
volume = {35},
number = {04},
doi = {10.1142/s0129065725500157}
}
EUVIMED