Cross-layer Attention Sharing for Pre-trained Large Language Models
Transactions of the Association for Computational Linguistics · 04.05.2026 · 14 · 656-688
DOI: 10.1162/tacl.a.616 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 04.05.2026. Verfasst von Yongyu Mu und 11 weiteren Autor:innen. Die Arbeit nennt 102 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
To enhance the efficiency of the attention mechanism within large language models (LLMs), previous works primarily compress the Key-Value cache or group attention heads, while largely overlooking redundancy between layers. Our comprehensive analyses across various LLMs show that highly similar attention patterns persist within most layers. It’s intuitive to reduce the redundancy by sharing attention weights across layers. However, further analysis reveals two challenges: (1) Directly sharing the weight matrix without carefully rearranging the attention heads proves to be ineffective; (2) Shallow layers are vulnerable to small deviations in attention weights. Driven by these insights, we introduce LiSA, a lightweight substitute for self-attention in well-trained LLMs. LiSA employs tiny feed-forward networks to align attention heads between adjacent layers and low-rank matrices to approximate differences in layer-wise attention weights. Evaluations encompassing 13 typical benchmarks demonstrate that LiSA maintains high response quality in terms of accuracy and perplexity while reducing redundant attention calculations within 53% −84% of the total layers. Our implementations of LiSA achieve a 6 × compression of Q and K matrices within the attention mechanism, with maximum throughput improvements 19.5%, 32.3%, and 40.1% for LLaMA3-8B, LLaMA2-7B, and LLaMA2-13B, respectively. Our code is available at https://github.com/takagi97/lisa.
Zitieren
APA
Mu, Y., Wu, Y., Fan, Y., Wang, C., Li, H., Zeng, J., He, Q., Yang, M., Meng, F., Zhou, J., Xiao, T., & Zhu, J. (2026). Cross-layer Attention Sharing for Pre-trained Large Language Models. Transactions of the Association for Computational Linguistics, 14, 656-688. https://doi.org/10.1162/tacl.a.616
Vancouver
Mu Y, Wu Y, Fan Y, Wang C, Li H, Zeng J, et al. Cross-layer Attention Sharing for Pre-trained Large Language Models. Transactions of the Association for Computational Linguistics. 2026;14:656-688. doi:10.1162/tacl.a.616
BibTeX
@article{Mu2026,
author = {Mu, Yongyu and Wu, Yuzhang and Fan, Yuchun and Wang, Chenglong and Li, Hengyu and Zeng, Jiali and He, Qiaozhi and Yang, Murun and Meng, Fandong and Zhou, Jie and Xiao, Tong and Zhu, Jingbo},
title = {Cross-layer Attention Sharing for Pre-trained Large Language Models},
journal = {Transactions of the Association for Computational Linguistics},
year = {2026},
volume = {14},
pages = {656--688},
doi = {10.1162/tacl.a.616}
}
EUVIMED