EUVIMEDEUVIMED entdecken
Crossref

A Comparative Study of KV-Cache Quantisation Methods for Large Language Model Inference

Chinmay C S

International Research Journal of Computer Science · 06.06.2026 · 13 (06) · 607

DOI: 10.26562/irjcs.2026.v1306.03 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Research Journal of Computer Science, online erschienen am 06.06.2026. Verfasst von Chinmay C S. Die Arbeit nennt 16 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A four-billion-parameter language model accumulates several gigabytes of key–value (KV) cache at a context of 32K tokens, often exceeding the size of the model weights. Quantising this cache reduces the memory footprint, but five published methods take different routes to that goal, with non-obvious trade-offs between them. This paper compares six KV-cache representations an uncompressed bf16 reference, KIVI, KVQuant, QuaRot, QJL, and Turbo Quant along five criteria including bit budget, calibration requirement, and decode-time key reconstruction. All numerical figures are taken from the corresponding source publications. Two findings follow. Reported compression ratios range from 2.6× (KIVI at 2-bit) to 6.4× (KVQuant at 2-bit non-uniform), and the gap between nominal bit budget and reported ratio reflects the cost of per-block metadata. QJL and TurboQuant share two properties absent from KIVI, KVQuant, and QuaRot: they require no calibration data, and they compute attention without reconstructing keys. This combination is most valuable in deployments where calibration prompts are unstable or unavailable, or where the same compressed cache spans multiple model variants.

Zitieren

APA

S, C. C. (2026). A Comparative Study of KV-Cache Quantisation Methods for Large Language Model Inference. International Research Journal of Computer Science, 13(06), 607. https://doi.org/10.26562/irjcs.2026.v1306.03

Vancouver

S CC. A Comparative Study of KV-Cache Quantisation Methods for Large Language Model Inference. International Research Journal of Computer Science. 2026;13(06):607. doi:10.26562/irjcs.2026.v1306.03

BibTeX
@article{S2026,
  author = {S, Chinmay C},
  title = {A Comparative Study of KV-Cache Quantisation Methods for Large Language Model Inference},
  journal = {International Research Journal of Computer Science},
  year = {2026},
  volume = {13},
  number = {06},
  pages = {607},
  doi = {10.26562/irjcs.2026.v1306.03}
}
Kostenlos starten →