Deadline-Constrained KV Admission Control for Mixed-Length Generative Inference
International Journal of Business and Social Science Studies · 29.08.2026 · 6 (2) · 99-109
DOI: 10.66795/ijbsss61 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Business and Social Science Studies, online erschienen am 29.08.2026. Verfasst von Eun-Seo Jang. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The rapid adoption of large language models in real-time applications has introduced unprecedented challenges in serving infrastructure, particularly concerning memory management and latency predictability. Autoregressive generative inference fundamentally relies on the Key-Value cache to prevent redundant computations, leading to severe memory bottlenecks when processing multiple requests concurrently. This paper investigates the critical problem of deadline-constrained admission control for mixed-length generative inference workloads. Traditional scheduling and admission control policies, which often assume deterministic execution times or uniform memory consumption, fail to address the highly variable nature of mixed-length sequences and the strict latency requirements of interactive applications. To mitigate these issues, we propose a novel deadline-constrained Key-Value admission control framework that integrates predictive memory allocation with deadline-aware scheduling mechanisms. By continuously monitoring the Key-Value cache utilization and probabilistically forecasting the output sequence lengths of incoming requests, the proposed system dynamically accepts or rejects requests to maximize overall system throughput while strictly adhering to user-defined latency constraints. Comprehensive theoretical modeling and subsequent analytical evaluations demonstrate that integrating deadline constraints directly into the Key-Value cache admission pipeline significantly reduces deadline miss rates compared to conventional first-come-first-serve or purely memory-based admission policies. The findings present a scalable approach to optimizing large language model serving systems, ensuring robust performance under high-concurrency, heterogeneous workload conditions.
Zitieren
APA
Jang, E. (2026). Deadline-Constrained KV Admission Control for Mixed-Length Generative Inference. International Journal of Business and Social Science Studies, 6(2), 99-109. https://doi.org/10.66795/ijbsss61
Vancouver
Jang E. Deadline-Constrained KV Admission Control for Mixed-Length Generative Inference. International Journal of Business and Social Science Studies. 2026;6(2):99-109. doi:10.66795/ijbsss61
BibTeX
@article{Jang2026,
author = {Jang, Eun-Seo},
title = {Deadline-Constrained KV Admission Control for Mixed-Length Generative Inference},
journal = {International Journal of Business and Social Science Studies},
year = {2026},
volume = {6},
number = {2},
pages = {99--109},
doi = {10.66795/ijbsss61}
}
EUVIMED