Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
ACM Transactions on Storage · 30.12.2026 · 22 (4) · 1-38
DOI: 10.1145/3773772 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Storage, online erschienen am 17.08.2026. Verfasst von Ruoyu Qin und 12 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research & Development Program of China, Beijing Natural Science Foundation, Tsinghua University Initiative Scientific Research Program.
Abstract (Originalsprache)
Mooncake is the serving platform for Kimi, an LLM chatbot service developed by Moonshot AI. This platform features a KVCache-centric disaggregated architecture that not only separates prefill and decoding clusters but also efficiently utilizes the underexploited CPU, DRAM, SSD and NIC resources of the GPU cluster to establish a disaggregated KVCache. At the core of Mooncake is its KVCache-centric global cache and a scheduler designed to maximize throughput while adhering to stringent latency-related Service Level Objectives (SLOs). Our experiments demonstrate that Mooncake excels in scenarios involving long-context inputs. In tests using real traces, Mooncake increases the effective request capacity by 59%∼498% when compared to baseline methods, all while complying with SLOs. Currently, Mooncake is operational across thousands of nodes, processing over 100 billion tokens daily. In practical deployments, Mooncake ’s innovative architecture enables Kimi to handle 115% and 107% more requests on NVIDIA A800 and H800 clusters, respectively, compared to previous systems.
Förderung
- National Key Research & Development Program of China
- Beijing Natural Science Foundation
- Tsinghua University Initiative Scientific Research Program
- Approaching.AI Co., Ltd.
Zitieren
APA
Qin, R., Li, Z., He, W., Cui, J., Tang, H., Ren, F., Ma, T., Cai, S., Zhang, Y., Zhang, M., Wu, Y., Zheng, W., & Xu, X. (2026). Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving. ACM Transactions on Storage, 22(4), 1-38. https://doi.org/10.1145/3773772
Vancouver
Qin R, Li Z, He W, Cui J, Tang H, Ren F, et al. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving. ACM Transactions on Storage. 2026;22(4):1-38. doi:10.1145/3773772
BibTeX
@article{Qin2026,
author = {Qin, Ruoyu and Li, Zheming and He, Weiran and Cui, Jialei and Tang, Heyi and Ren, Feng and Ma, Teng and Cai, Shangming and Zhang, Yineng and Zhang, Mingxing and Wu, Yongwei and Zheng, Weimin and Xu, Xinran},
title = {Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving},
journal = {ACM Transactions on Storage},
year = {2026},
volume = {22},
number = {4},
pages = {1--38},
doi = {10.1145/3773772}
}
EUVIMED