DS2 Attention: Dual-Stream Segmented Information Propagating Linear Attention for Vision Transformers
AI · 24.05.2026 · 7 (6) · 188
DOI: 10.3390/ai7060188 ↗Kurz gefasst
Fachartikel in der Zeitschrift AI, online erschienen am 24.05.2026. Verfasst von Rigel Mahmood und 2 weiteren Autor:innen. Die Arbeit nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
While Vision Transformers (ViTs) have achieved state-of-the-art (SOTA) results in visual recognition, their scalability remains fundamentally constrained by the quadratic complexity of global self-attention. To address this, we present a linear complexity attention design employing dual-stream information propagation to enhance representational efficiency and structured feature aggregation. Our proposed DS2 attention acts as a versatile replacement for standard attention in various SOTA designs, such as Tokens-to-Token (T2T) and FasterViT. In our design, half of the attention heads perform left-to-right segmented information propagation in a Perceiver-style manner, while the remaining half of the heads perform right-to-left propagation. This bidirectional structured attention enables efficient long-range dependency modeling without the overhead of full global attention. To improve classification performance, we introduce a segment-level classification strategy in which each segment is associated with a summary token. The final prediction is produced via cross-attention between image tokens and these summary tokens, enabling hierarchical semantic comprehension. Extensive experiments demonstrate that the proposed attention design achieves on average 0.3% higher accuracy on the ImageNet-1K dataset, while offering improved information flow and higher efficiency across SOTA Vision Transformer designs.
Zitieren
APA
Mahmood, R., Patel, S., & Elleithy, K. (2026). DS2 Attention: Dual-Stream Segmented Information Propagating Linear Attention for Vision Transformers. AI, 7(6), 188. https://doi.org/10.3390/ai7060188
Vancouver
Mahmood R, Patel S, Elleithy K. DS2 Attention: Dual-Stream Segmented Information Propagating Linear Attention for Vision Transformers. AI. 2026;7(6):188. doi:10.3390/ai7060188
BibTeX
@article{Mahmood2026,
author = {Mahmood, Rigel and Patel, Sarosh and Elleithy, Khaled},
title = {DS2 Attention: Dual-Stream Segmented Information Propagating Linear Attention for Vision Transformers},
journal = {AI},
year = {2026},
volume = {7},
number = {6},
pages = {188},
doi = {10.3390/ai7060188}
}
EUVIMED