QubitCache: Quantum-Inspired Probabilistic Attention Preservation for KV-Cache Compression
IEEE Access · 2026 · 14 · 56679-56693
DOI: 10.1109/access.2026.3680126 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Jieui Kang und 3 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Research Foundation of Korea, Korean Government [Ministry of Science and ICT (MSIT)], Institute of Information and Communications Technology Planning and Evaluation (IITP)-Information Technology Research Center.
Abstract (Originalsprache)
Large language model inference faces a critical memory bottleneck from KV cache, which grows linearly with sequence length and dominates GPU memory during long-context generation. Existing compression methods reduce memory through token eviction but irreversibly discard attention relationships essential for complex reasoning. We present QubitCache, a framework that reframes KV cache compression as attention distribution compression rather than token selection. Our approach employs a quantum-inspired mathematical formalism (using the hierarchical amplitude encoding structure from quantum computing as a classical probability distribution compressor, without requiring quantum hardware) to represent attention weight distributions in logarithmic parameter space. QubitCache introduces a hybrid architecture: critical tokens (15% of sequence) remain in full-precision classical storage, while attention distributions over remaining tokens are compressed using hierarchical amplitude encoding, achieving a $57\times $ parameter reduction per segment. We provide formal analysis showing that the pipeline output error is bounded by the fraction of attention mass assigned to compressed tokens, which is small ( $\leq 15$ %) due to inherent attention sparsity (Theorem 1). An important property of our design is that the compressed distribution is fixed after prefill and is therefore query-agnostic during decoding; we analyze the implications of this trade-off and provide empirical evidence that the resulting $\lambda _{\text {mass}}$ stability is sufficient for the tasks evaluated. During inference, compressed distributions are reconstructed to provide soft attention weights, maintaining contextual influence of compressed tokens through probabilistic rather than binary decisions. We evaluate QubitCache on four long-context models across sixteen LongBench tasks and the RULER benchmark, comparing against both classical baselines (H2O, SnapKV) and recent methods (MiniKV, Compactor). QubitCache achieves approximately $6.7\times $ memory reduction while retaining 91–99% of full-KV performance, with an average of 74.8 on RULER, which is 10.5 points above token-eviction baselines. Latency analysis confirms that per-token decode overhead remains within $2\times $ of eviction-based methods. Comprehensive analysis including sensitivity, quantization integration, and failure cases characterizes when this approach succeeds and where it has limitations.
Förderung
- National Research Foundation of Korea
- Korean Government [Ministry of Science and ICT (MSIT)]
- Institute of Information and Communications Technology Planning and Evaluation (IITP)-Information Technology Research Center
- Korean Government
Zitieren
APA
Kang, J., Choi, J., Noh, W., & Sim, J. (2026). QubitCache: Quantum-Inspired Probabilistic Attention Preservation for KV-Cache Compression. IEEE Access, 14, 56679-56693. https://doi.org/10.1109/access.2026.3680126
Vancouver
Kang J, Choi J, Noh W, Sim J. QubitCache: Quantum-Inspired Probabilistic Attention Preservation for KV-Cache Compression. IEEE Access. 2026;14:56679-56693. doi:10.1109/access.2026.3680126
BibTeX
@article{Kang2026,
author = {Kang, Jieui and Choi, Jaeyoung and Noh, Wonhui and Sim, Jaehyeong},
title = {QubitCache: Quantum-Inspired Probabilistic Attention Preservation for KV-Cache Compression},
journal = {IEEE Access},
year = {2026},
volume = {14},
pages = {56679--56693},
doi = {10.1109/access.2026.3680126}
}
EUVIMED