Efficient Multimodal Large Language Model via Dynamic KV Cache Quantization
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (25) · 20994-21001
DOI: 10.1609/aaai.v40i25.39241 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jiahao Fan und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Multimodal large language models (LMMs) have demonstrated remarkable capabilities across diverse vision-language tasks, including image captioning, visual question answering, and text-image retrieval. However, their computational complexity and memory footprint, particularly in the key-value (KV) cache during inference, pose significant challenges for real-time deployment, especially on resource-constrained devices. In this paper, we propose Dynamic KV Cache Quantization, a novel quantization strategy tailored for multimodal LMMs. Our approach applies per-channel quantization to (K) and per-token quantization to (V), leveraging their respective statistical distributions to optimize precision allocation. Additionally, we introduce an adaptive token and channel recording mechanism that dynamically adjusts quantization parameters based on real-time distribution tracking, effectively mitigating the impact of outliers. To further enhance compression efficiency, we implement fine-grained grouping, which partitions KV tensors into localized subgroups, enabling more adaptive quantization. Experimental results on LLaVA-1.5 (7B/13B) and Qwen-VL across multiple multimodal benchmarks demonstrate that our method significantly outperforms existing KV-cache quantization approaches, achieving a superior trade-off between memory efficiency and model accuracy.
Zitieren
APA
Fan, J., & Chen, C. (2026). Efficient Multimodal Large Language Model via Dynamic KV Cache Quantization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(25), 20994-21001. https://doi.org/10.1609/aaai.v40i25.39241
Vancouver
Fan J, Chen C. Efficient Multimodal Large Language Model via Dynamic KV Cache Quantization. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(25):20994-21001. doi:10.1609/aaai.v40i25.39241
BibTeX
@article{Fan2026,
author = {Fan, Jiahao and Chen, Chien-Ming},
title = {Efficient Multimodal Large Language Model via Dynamic KV Cache Quantization},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {25},
pages = {20994--21001},
doi = {10.1609/aaai.v40i25.39241}
}
EUVIMED