Multi-Context Concatenation Across Requests for LLMs
Axioms · 22.04.2026 · 15 (5) · 303
DOI: 10.3390/axioms15050303 ↗Kurz gefasst
Fachartikel in der Zeitschrift Axioms, online erschienen am 22.04.2026. Verfasst von Ziyi Cao und 5 weiteren Autor:innen. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Shandong Province, State Key Laboratory of Processors.
Abstract (Originalsprache)
Reusing separate, pre-filled Key-Value (KV) Caches for multiple contexts has become a common practice in handling multi-context scenarios with Large Language Models. However, this leads to a lack of cross-attention mechanisms between contexts. To address this, we propose CatLLM, the first method that concatenates multiple contexts across requests offline to compensate for this deficiency. Specifically, during offline processing, CatLLM identifies contexts that severely lack cross-attention by incorporating the weighted inner products of Q and K vectors from tokens in an un-concatenated context into an equivalently transformed weighted formulation for concatenated Q and K inner products. This yields a weighting wiA+B corresponding to the output vector difference, which can then be used to identify contexts with severe cross-attention deficiencies and concatenate them into a single context for KV Cache computation. Experimental results show that, compared to the baseline of separate caching (i.e., no concatenation), fully concatenating all contexts improves the F1 score by 6%. Meanwhile, the proposed method reduces the number of contexts requiring caching from 10 to 7 while achieving a 3% F1 score, thereby maximizing performance improvement while minimizing the degree of context compression.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Shandong Province
- State Key Laboratory of Processors
Zitieren
APA
Cao, Z., Zhang, J., Wang, S., Li, S., Liu, B., & Xie, H. (2026). Multi-Context Concatenation Across Requests for LLMs. Axioms, 15(5), 303. https://doi.org/10.3390/axioms15050303
Vancouver
Cao Z, Zhang J, Wang S, Li S, Liu B, Xie H. Multi-Context Concatenation Across Requests for LLMs. Axioms. 2026;15(5):303. doi:10.3390/axioms15050303
BibTeX
@article{Cao2026,
author = {Cao, Ziyi and Zhang, Jingbin and Wang, Shuai and Li, Shaobo and Liu, Bingquan and Xie, Heng},
title = {Multi-Context Concatenation Across Requests for LLMs},
journal = {Axioms},
year = {2026},
volume = {15},
number = {5},
pages = {303},
doi = {10.3390/axioms15050303}
}
EUVIMED