EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference

Shenglong Liu, Yanli Lv, Siyao An, Chenghuan Yu, Yiwei Ru

Electronics · 21.07.2026 · 15 (14) · 3206

DOI: 10.3390/electronics15143206 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 21.07.2026. Verfasst von Shenglong Liu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: State Grid Information & Telecommunication Center.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal large language models (MLLMs) face substantial memory bottlenecks when processing long visual contexts, such as videos and high-resolution images. Existing methods that allocate visual KV cache budgets using cross-modal attention entropy mainly estimate the distributional breadth of text–vision interaction and may overlook how visual relevance varies across query positions in the encoded multimodal context. We propose a cross-modal query-position variance-aware KV cache optimization method for efficient multimodal long-context inference. The proposed method combines cross-modal attention entropy with prefill-stage query-position variance computed from cross-modal attention to estimate layer-wise visual KV cache preferences. Based on this preference score, visual KV cache budgets are allocated across layers, and variance-aware token pruning is applied to retain high-importance KV states while directly evicting redundant visual tokens without feature merging. Experiments on the MileBench benchmark using LLaVA-v1.5-7B show that, under deterministic single-run evaluation and while retaining only 20% of the visual KV cache, the proposed method produces point-estimate performance close to the full-cache reference and higher point estimates on several fine-grained reasoning and retrieval subtasks. Additional representative-subtask evaluations under different visual cache budgets and on InternVL2.5-8B further provide preliminary point-estimate evidence that the proposed allocation signal is not restricted to a single cache ratio or backbone. System profiling further shows that the 20% cache setting reduces measured KV cache GPU memory from 1.28 GiB to 0.26 GiB and decoding latency from 100.28 ms/token to 92.85 ms/token. These results suggest that cross-modal query-position variance may help preserve sparse, query-dependent visual cues under low-cache-budget multimodal inference.

Förderung

  • State Grid Information & Telecommunication Center

Zitieren

APA

Liu, S., Lv, Y., An, S., Yu, C., & Ru, Y. (2026). Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference. Electronics, 15(14), 3206. https://doi.org/10.3390/electronics15143206

Vancouver

Liu S, Lv Y, An S, Yu C, Ru Y. Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference. Electronics. 2026;15(14):3206. doi:10.3390/electronics15143206

BibTeX
@article{Liu2026,
  author = {Liu, Shenglong and Lv, Yanli and An, Siyao and Yu, Chenghuan and Ru, Yiwei},
  title = {Cross-Modal Variance-Aware KV Cache Optimization for Efficient Multimodal Long-Context Inference},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {14},
  pages = {3206},
  doi = {10.3390/electronics15143206}
}
Kostenlos starten →