Efficient LLM Serving Under Variable Cloud Traffic Loads
Dandao Xuebao/Journal of Ballistics · 23.05.2026 · 38 (1) · 438-449
DOI: 10.52783/dxjb.v38.293 ↗Kurz gefasst
Fachartikel in der Zeitschrift Dandao Xuebao/Journal of Ballistics, online erschienen am 23.05.2026. Verfasst von MD Faruk Abdulla.
Abstract (Originalsprache)
Large Language Models (LLMs) have rapidly become core components of modern cloud-based applications, powering services ranging from conversational agents to automated code generation. However, serving LLMs efficiently under unpredictable and variable traffic conditions remains a critical open challenge. Unlike traditional web services, LLM inference is computationally intensive, memory-bound, and highly sensitive to request length variability, making conventional auto-scaling strategies insufficient. This paper presents AdaptServe, a cloud-native LLM serving system designed to handle dynamic traffic loads through intelligent request batching, predictive scaling, and model-tiering strategies. AdaptServe employs a traffic-aware scheduler that continuously monitors incoming request patterns and adjusts GPU resource allocation in real time. Evaluated on real-world traffic traces from production LLM deployments, AdaptServe achieves a 4.2x improvement in throughput, 58% reduction in tail latency, and 34% decrease in cloud serving costs compared to static deployment baselines, while maintaining response quality above acceptable thresholds.
Zitieren
APA
Abdulla, M. F. (2026). Efficient LLM Serving Under Variable Cloud Traffic Loads. Dandao Xuebao/Journal of Ballistics, 38(1), 438-449. https://doi.org/10.52783/dxjb.v38.293
Vancouver
Abdulla MF. Efficient LLM Serving Under Variable Cloud Traffic Loads. Dandao Xuebao/Journal of Ballistics. 2026;38(1):438-449. doi:10.52783/dxjb.v38.293
BibTeX
@article{Abdulla2026,
author = {Abdulla, MD Faruk},
title = {Efficient LLM Serving Under Variable Cloud Traffic Loads},
journal = {Dandao Xuebao/Journal of Ballistics},
year = {2026},
volume = {38},
number = {1},
pages = {438--449},
doi = {10.52783/dxjb.v38.293}
}
EUVIMED