EUVIMEDEUVIMED entdecken
Crossref

Efficient LLM Serving Under Variable Cloud Traffic Loads

MD Faruk Abdulla

Dandao Xuebao/Journal of Ballistics · 23.05.2026 · 38 (1) · 438-449

DOI: 10.52783/dxjb.v38.293 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Dandao Xuebao/Journal of Ballistics, online erschienen am 23.05.2026. Verfasst von MD Faruk Abdulla.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) have rapidly become core components of modern cloud-based applications, powering services ranging from conversational agents to automated code generation. However, serving LLMs efficiently under unpredictable and variable traffic conditions remains a critical open challenge. Unlike traditional web services, LLM inference is computationally intensive, memory-bound, and highly sensitive to request length variability, making conventional auto-scaling strategies insufficient. This paper presents AdaptServe, a cloud-native LLM serving system designed to handle dynamic traffic loads through intelligent request batching, predictive scaling, and model-tiering strategies. AdaptServe employs a traffic-aware scheduler that continuously monitors incoming request patterns and adjusts GPU resource allocation in real time. Evaluated on real-world traffic traces from production LLM deployments, AdaptServe achieves a 4.2x improvement in throughput, 58% reduction in tail latency, and 34% decrease in cloud serving costs compared to static deployment baselines, while maintaining response quality above acceptable thresholds.

Zitieren

APA

Abdulla, M. F. (2026). Efficient LLM Serving Under Variable Cloud Traffic Loads. Dandao Xuebao/Journal of Ballistics, 38(1), 438-449. https://doi.org/10.52783/dxjb.v38.293

Vancouver

Abdulla MF. Efficient LLM Serving Under Variable Cloud Traffic Loads. Dandao Xuebao/Journal of Ballistics. 2026;38(1):438-449. doi:10.52783/dxjb.v38.293

BibTeX
@article{Abdulla2026,
  author = {Abdulla, MD Faruk},
  title = {Efficient LLM Serving Under Variable Cloud Traffic Loads},
  journal = {Dandao Xuebao/Journal of Ballistics},
  year = {2026},
  volume = {38},
  number = {1},
  pages = {438--449},
  doi = {10.52783/dxjb.v38.293}
}
Kostenlos starten →