EUVIMEDEUVIMED entdecken
Crossref

ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models

Jianxin Yan, Wangze Ni, Lei Chen, Xuemin Lin, Peng Cheng, Zhan Qin, Kui Ren

Proceedings of the VLDB Endowment · 2025-08 · 18 (12) · 5391-5394

DOI: 10.14778/3750601.3750679 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 16.09.2025. Verfasst von Jianxin Yan und 6 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 8 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Semantic caching significantly reduces computational costs and improves efficiency by storing and reusing large language model (LLM) responses. However, existing systems rely primarily on matching individual queries, lacking awareness of multi-turn dialogue contexts, which leads to incorrect cache hits when similar queries appear in different conversational settings. This demonstration introduces ContextCache, a context-aware semantic caching system for multi-turn dialogues. ContextCache employs a two-stage retrieval architecture that first executes vector-based retrieval on the current query to identify potential matches and then integrates current and historical dialogue representations through self-attention mechanisms for precise contextual matching. Evaluation of real-world conversations shows that ContextCache improves precision and recall compared to existing methods. Additionally, cached responses exhibit approximately 10 times lower latency than direct LLM invocation, enabling significant computational cost reductions for LLM conversational applications.

Zitieren

APA

Yan, J., Ni, W., Chen, L., Lin, X., Cheng, P., Qin, Z., & Ren, K. (2025). ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models. Proceedings of the VLDB Endowment, 18(12), 5391-5394. https://doi.org/10.14778/3750601.3750679

Vancouver

Yan J, Ni W, Chen L, Lin X, Cheng P, Qin Z, et al. ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models. Proceedings of the VLDB Endowment. 2025;18(12):5391-5394. doi:10.14778/3750601.3750679

BibTeX
@article{Yan2025,
  author = {Yan, Jianxin and Ni, Wangze and Chen, Lei and Lin, Xuemin and Cheng, Peng and Qin, Zhan and Ren, Kui},
  title = {ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models},
  journal = {Proceedings of the VLDB Endowment},
  year = {2025},
  volume = {18},
  number = {12},
  pages = {5391--5394},
  doi = {10.14778/3750601.3750679}
}
Kostenlos starten →