PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-11 · 48 (11) · 15412-15426
DOI: 10.1109/tpami.2026.3723414 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-11. Verfasst von Weikang Meng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 85 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Linear attention has recently emerged as an efficient alternative to Softmax-based self-attention by reducing quadratic computational complexity to linear. However, existing linear attention mechanisms still suffer from substantial performance degradation, primarily due to two limitations: the loss of negative elements in query and key vectors caused by non-negative feature maps, and the failure to preserve the spikiness property as in Softmax attention, which weakens token discriminability. To address these issues, we propose PolaFormer++, an enhanced polarity-aware linear attention mechanism with Channel-wise Spikiness. Specifically, we first introduce the polarity-aware linear attention to recover both same- and opposite-polarity interactions via polarity decomposition, preserving complete similarity information. Secondly, we propose a Polarity-aware Channel-wise Spiky (PaCS) feature map, which incorporates two learnable temperature vectors into the dual polarity flows to endow each channel with a distinct degree of spikiness, thereby enabling adaptive control of channel-wise attention across polarity flows. Moreover, we provide the first theoretical analysis of how channel-wise feature maps influence the entropy of attention weight, revealing conditions under which a feature map preserves entropy-reducing behavior. Comprehensive experiments on image classification, object detection, semantic segmentation, image super-resolution, diffusion-based generation, and 3D novel view synthesis demonstrate that PolaFormer++ achieves a superior balance between efficiency and accuracy, establishing a strong theoretical and practical foundation for linear attention in vision Transformers.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Meng, W., Luo, Y., Huo, L., Li, Y., Wang, Y., Zhang, Z., & Shen, H. T. (2026). PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(11), 15412-15426. https://doi.org/10.1109/tpami.2026.3723414
Vancouver
Meng W, Luo Y, Huo L, Li Y, Wang Y, Zhang Z, et al. PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(11):15412-15426. doi:10.1109/tpami.2026.3723414
BibTeX
@article{Meng2026,
author = {Meng, Weikang and Luo, Yadan and Huo, Liangyu and Li, Yingjian and Wang, Yaowei and Zhang, Zheng and Shen, Heng Tao},
title = {PolaFormer++: Polarity-Aware Linear Attention With Channel-Wise Spikiness},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {11},
pages = {15412--15426},
doi = {10.1109/tpami.2026.3723414}
}
EUVIMED