EUVIMEDEUVIMED entdecken
Crossref

PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness

Weikang Meng, Yadan Luo, Liangyu Huo, Yingjian Li, Yaowei Wang, Zheng Zhang, Heng Tao Shen

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-11 · 48 (11) · 15412-15426

DOI: 10.1109/tpami.2026.3723414 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-11. Verfasst von Weikang Meng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 85 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Linear attention has recently emerged as an efficient alternative to Softmax-based self-attention by reducing quadratic computational complexity to linear. However, existing linear attention mechanisms still suffer from substantial performance degradation, primarily due to two limitations: the loss of negative elements in query and key vectors caused by non-negative feature maps, and the failure to preserve the spikiness property as in Softmax attention, which weakens token discriminability. To address these issues, we propose PolaFormer++, an enhanced polarity-aware linear attention mechanism with Channel-wise Spikiness. Specifically, we first introduce the polarity-aware linear attention to recover both same- and opposite-polarity interactions via polarity decomposition, preserving complete similarity information. Secondly, we propose a Polarity-aware Channel-wise Spiky (PaCS) feature map, which incorporates two learnable temperature vectors into the dual polarity flows to endow each channel with a distinct degree of spikiness, thereby enabling adaptive control of channel-wise attention across polarity flows. Moreover, we provide the first theoretical analysis of how channel-wise feature maps influence the entropy of attention weight, revealing conditions under which a feature map preserves entropy-reducing behavior. Comprehensive experiments on image classification, object detection, semantic segmentation, image super-resolution, diffusion-based generation, and 3D novel view synthesis demonstrate that PolaFormer++ achieves a superior balance between efficiency and accuracy, establishing a strong theoretical and practical foundation for linear attention in vision Transformers.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Meng, W., Luo, Y., Huo, L., Li, Y., Wang, Y., Zhang, Z., & Shen, H. T. (2026). PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(11), 15412-15426. https://doi.org/10.1109/tpami.2026.3723414

Vancouver

Meng W, Luo Y, Huo L, Li Y, Wang Y, Zhang Z, et al. PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(11):15412-15426. doi:10.1109/tpami.2026.3723414

BibTeX
@article{Meng2026,
  author = {Meng, Weikang and Luo, Yadan and Huo, Liangyu and Li, Yingjian and Wang, Yaowei and Zhang, Zheng and Shen, Heng Tao},
  title = {PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {11},
  pages = {15412--15426},
  doi = {10.1109/tpami.2026.3723414}
}
Kostenlos starten →