EUVIMEDEUVIMED entdecken
Crossref

RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation

Chao Jin, Zili Zhang, Xuanlin Jiang, Fangyue Liu, Shufan Liu, Xuanzhe Liu, Xin Jin

ACM Transactions on Computer Systems · 28.02.2026 · 44 (1) · 1-27

DOI: 10.1145/3768628 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Computer Systems, online erschienen am 07.11.2025. Verfasst von Chao Jin und 6 weiteren Autor:innen. Die Arbeit wurde bisher 64-mal zitiert (Crossref) und nennt 71 Literaturangaben. Gefördert von: National Key Research and Development Program of China, Scientific Research Innovation Capability Support Project for Young Faculty, Fundamental Research Funds for the Central Universities, Peking University, and the National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Retrieval-Augmented Generation (RAG) has demonstrated substantial advancements in various natural language processing tasks by integrating the strengths of large language models (LLMs) and external knowledge databases. However, the retrieval step introduces long sequence generation and extra data dependency, resulting in long end-to-end latency. Our analysis benchmarks current RAG systems and reveals that, while the retrieval step poses performance challenges, it also offers optimization opportunities through its retrieval pattern and streaming search behavior. We propose RAGCache, a latency-optimized serving system tailored for RAG. RAGCache leverages the retrieval pattern to organize and cache the intermediate states of retrieved knowledge in a knowledge tree across the GPU and host memory hierarchy, reducing LLM generation time. RAGCache employs dynamic speculative pipelining to exploit the streaming search behavior, overlapping retrieval with LLM generation to minimize end-to-end latency. We implement RAGCache based on vLLM and Faiss, and evaluate it on both open-source and production datasets. Experimental results demonstrate that RAGCache reduces the time to first token (TTFT) by up to 4× and improves the throughput by up to 2.1× compared to vLLM integrated with Faiss.

Förderung

  • National Key Research and Development Program of China
  • Scientific Research Innovation Capability Support Project for Young Faculty
  • Fundamental Research Funds for the Central Universities, Peking University, and the National Natural Science Foundation of China

Zitieren

APA

Jin, C., Zhang, Z., Jiang, X., Liu, F., Liu, S., Liu, X., & Jin, X. (2026). RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation. ACM Transactions on Computer Systems, 44(1), 1-27. https://doi.org/10.1145/3768628

Vancouver

Jin C, Zhang Z, Jiang X, Liu F, Liu S, Liu X, et al. RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation. ACM Transactions on Computer Systems. 2026;44(1):1-27. doi:10.1145/3768628

BibTeX
@article{Jin2026,
  author = {Jin, Chao and Zhang, Zili and Jiang, Xuanlin and Liu, Fangyue and Liu, Shufan and Liu, Xuanzhe and Jin, Xin},
  title = {RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation},
  journal = {ACM Transactions on Computer Systems},
  year = {2026},
  volume = {44},
  number = {1},
  pages = {1--27},
  doi = {10.1145/3768628}
}
Kostenlos starten →