Efficient Cooperation-Aware Key and Value Management for LLM Inference
Proceedings of the VLDB Endowment · 01.05.2026 · 19 (9) · 2154-2167
DOI: 10.14778/3819518.3819541 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 19.08.2026. Verfasst von Qiheng Sun und 6 weiteren Autor:innen. Die Arbeit nennt 58 Literaturangaben.
Abstract (Originalsprache)
Key-value (KV) caching is a widely used technique for boosting performance in database and storage systems. It keeps frequently accessed data in fast storage to minimize redundant data fetching and improve throughput. This same idea has been adopted in Large Language Models (LLMs), where it avoids recomputing the key and value states of previous tokens in attention heads during autoregressive decoding, thereby greatly accelerating inference. However, the KV cache in LLMs faces a significant challenge due to the substantial memory required to store these KV pairs in the inference process. This issue arises because each attention head in the LLM stores its own KV cache for all context tokens, leading to the cache size that grows linearly with sequence length. This has spurred research into efficient management of the KV cache of LLMs. One of the promising directions is KV cache budget allocation, with several approaches proposing head-level allocation as they recognize that different attention heads play distinct roles. However, these methods assess each head in isolation, overlooking their cooperative contributions within the model, which results in a deviation from their true impact. To address this limitation, we propose CoKV, a novel method that efficiently manages the KV cache in LLM inference by modeling the cooperation among attention heads as a cooperative game. By attributing the contribution of each head within the model in advance, CoKV can more effectively allocate the global KV cache budget in KV cache optimization techniques such as eviction and quantization. Extensive experiments demonstrate the effectiveness of CoKV on long-context benchmarks (e.g., LongBench, NIAH, and RULER) and mathematical reasoning benchmarks (e.g., GSM8K and MATH) across multiple model families, including Qwen, Llama, and Mistral.
Zitieren
APA
Sun, Q., Zhang, H., Liu, J., Xia, H., Liu, J., Ren, K., & Hu, H. (2026). Efficient Cooperation-Aware Key and Value Management for LLM Inference. Proceedings of the VLDB Endowment, 19(9), 2154-2167. https://doi.org/10.14778/3819518.3819541
Vancouver
Sun Q, Zhang H, Liu J, Xia H, Liu J, Ren K, et al. Efficient Cooperation-Aware Key and Value Management for LLM Inference. Proceedings of the VLDB Endowment. 2026;19(9):2154-2167. doi:10.14778/3819518.3819541
BibTeX
@article{Sun2026,
author = {Sun, Qiheng and Zhang, Hongwei and Liu, Junxu and Xia, Haocheng and Liu, Jinfei and Ren, Kui and Hu, Haibo},
title = {Efficient Cooperation-Aware Key and Value Management for LLM Inference},
journal = {Proceedings of the VLDB Endowment},
year = {2026},
volume = {19},
number = {9},
pages = {2154--2167},
doi = {10.14778/3819518.3819541}
}
EUVIMED