RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
ACM Transactions on Computer Systems · 28.02.2026 · 44 (1) · 1-27
DOI: 10.1145/3768628 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Computer Systems, online erschienen am 07.11.2025. Verfasst von Chao Jin und 6 weiteren Autor:innen. Die Arbeit wurde bisher 64-mal zitiert (Crossref) und nennt 71 Literaturangaben. Gefördert von: National Key Research and Development Program of China, Scientific Research Innovation Capability Support Project for Young Faculty, Fundamental Research Funds for the Central Universities, Peking University, and the National Natural Science Foundation of China.
Abstract (Originalsprache)
Retrieval-Augmented Generation (RAG) has demonstrated substantial advancements in various natural language processing tasks by integrating the strengths of large language models (LLMs) and external knowledge databases. However, the retrieval step introduces long sequence generation and extra data dependency, resulting in long end-to-end latency. Our analysis benchmarks current RAG systems and reveals that, while the retrieval step poses performance challenges, it also offers optimization opportunities through its retrieval pattern and streaming search behavior. We propose RAGCache, a latency-optimized serving system tailored for RAG. RAGCache leverages the retrieval pattern to organize and cache the intermediate states of retrieved knowledge in a knowledge tree across the GPU and host memory hierarchy, reducing LLM generation time. RAGCache employs dynamic speculative pipelining to exploit the streaming search behavior, overlapping retrieval with LLM generation to minimize end-to-end latency. We implement RAGCache based on vLLM and Faiss, and evaluate it on both open-source and production datasets. Experimental results demonstrate that RAGCache reduces the time to first token (TTFT) by up to 4× and improves the throughput by up to 2.1× compared to vLLM integrated with Faiss.
Förderung
- National Key Research and Development Program of China
- Scientific Research Innovation Capability Support Project for Young Faculty
- Fundamental Research Funds for the Central Universities, Peking University, and the National Natural Science Foundation of China
Zitieren
APA
Jin, C., Zhang, Z., Jiang, X., Liu, F., Liu, S., Liu, X., & Jin, X. (2026). RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation. ACM Transactions on Computer Systems, 44(1), 1-27. https://doi.org/10.1145/3768628
Vancouver
Jin C, Zhang Z, Jiang X, Liu F, Liu S, Liu X, et al. RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation. ACM Transactions on Computer Systems. 2026;44(1):1-27. doi:10.1145/3768628
BibTeX
@article{Jin2026,
author = {Jin, Chao and Zhang, Zili and Jiang, Xuanlin and Liu, Fangyue and Liu, Shufan and Liu, Xuanzhe and Jin, Xin},
title = {RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation},
journal = {ACM Transactions on Computer Systems},
year = {2026},
volume = {44},
number = {1},
pages = {1--27},
doi = {10.1145/3768628}
}
EUVIMED