Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference
Electronics · 21.07.2026 · 15 (14) · 3206
DOI: 10.3390/electronics15143206 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 21.07.2026. Verfasst von Shenglong Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: State Grid Information & Telecommunication Center.
Abstract (Originalsprache)
Multimodal large language models (MLLMs) face substantial memory bottlenecks when processing long visual contexts, such as videos and high-resolution images. Existing methods that allocate visual KV cache budgets using cross-modal attention entropy mainly estimate the distributional breadth of text–vision interaction and may overlook how visual relevance varies across query positions in the encoded multimodal context. We propose a cross-modal query-position variance-aware KV cache optimization method for efficient multimodal long-context inference. The proposed method combines cross-modal attention entropy with prefill-stage query-position variance computed from cross-modal attention to estimate layer-wise visual KV cache preferences. Based on this preference score, visual KV cache budgets are allocated across layers, and variance-aware token pruning is applied to retain high-importance KV states while directly evicting redundant visual tokens without feature merging. Experiments on the MileBench benchmark using LLaVA-v1.5-7B show that, under deterministic single-run evaluation and while retaining only 20% of the visual KV cache, the proposed method produces point-estimate performance close to the full-cache reference and higher point estimates on several fine-grained reasoning and retrieval subtasks. Additional representative-subtask evaluations under different visual cache budgets and on InternVL2.5-8B further provide preliminary point-estimate evidence that the proposed allocation signal is not restricted to a single cache ratio or backbone. System profiling further shows that the 20% cache setting reduces measured KV cache GPU memory from 1.28 GiB to 0.26 GiB and decoding latency from 100.28 ms/token to 92.85 ms/token. These results suggest that cross-modal query-position variance may help preserve sparse, query-dependent visual cues under low-cache-budget multimodal inference.
Förderung
- State Grid Information & Telecommunication Center
Zitieren
APA
Liu, S., Lv, Y., An, S., Yu, C., & Ru, Y. (2026). Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference. Electronics, 15(14), 3206. https://doi.org/10.3390/electronics15143206
Vancouver
Liu S, Lv Y, An S, Yu C, Ru Y. Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference. Electronics. 2026;15(14):3206. doi:10.3390/electronics15143206
BibTeX
@article{Liu2026,
author = {Liu, Shenglong and Lv, Yanli and An, Siyao and Yu, Chenghuan and Ru, Yiwei},
title = {Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference},
journal = {Electronics},
year = {2026},
volume = {15},
number = {14},
pages = {3206},
doi = {10.3390/electronics15143206}
}
EUVIMED