SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (29) · 24124-24132
DOI: 10.1609/aaai.v40i29.39591 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yu-Chen Lu und 7 weiteren Autor:innen.
Abstract (Originalsprache)
Large language models (LLM) have achieved remarkable performance across a wide range of tasks. However, their substantial parameter sizes pose significant challenges for deployment on edge devices with limited computational and memory resources. Low-rank compression is a promising approach to address this issue, as it reduces both computational and memory costs, making LLM more suitable for resource-constrained environments. Nonetheless, naïve low-rank compression methods require a significant reduction in the retained rank to achieve meaningful memory and computation savings. For a low-rank model, the ranks need to be reduced by more than half to yield efficiency gains. Such aggressive truncation, however, typically results in substantial performance degradation. To address this trade-off, we propose SkipCat, a novel low-rank compression framework that enables the use of higher ranks while achieving the same compression rates. First, we introduce an intra-layer shared low-rank projection method, where multiple matrices that share the same input use a common projection. This reduces redundancy and improves compression efficiency. Second, we propose a block skipping technique that omits computations and memory transfers for selected sub-blocks within the low-rank decomposition. These two techniques jointly enable our compressed model to retain more effective ranks under the same compression budget. Experimental results show that, without any additional fine-tuning, our method outperforms previous low-rank compression approaches by 7% accuracy improvement on zero-shot tasks under the same compression rate. These results highlight the effectiveness of our rank-maximized compression strategy in preserving model performance under tight resource constraints.
Zitieren
APA
Lu, Y., Yu, S., Weng, H., Wang, P., Hu, Y., Hung-Chun, L., Chiang, H., & Wu, K. (2026). SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping. Proceedings of the AAAI Conference on Artificial Intelligence, 40(29), 24124-24132. https://doi.org/10.1609/aaai.v40i29.39591
Vancouver
Lu Y, Yu S, Weng H, Wang P, Hu Y, Hung-Chun L, et al. SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(29):24124-24132. doi:10.1609/aaai.v40i29.39591
BibTeX
@article{Lu2026,
author = {Lu, Yu-Chen and Yu, Sheng-Feng and Weng, Hui-Hsien and Wang, Pei-Shuo and Hu, Yu-Fang and Hung-Chun, Liang and Chiang, Hung-Yueh and Wu, Kai-Chiang},
title = {SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skipping},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {29},
pages = {24124--24132},
doi = {10.1609/aaai.v40i29.39591}
}
EUVIMED