EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Risk-Aware LLM Serving with Multi-Horizon Demand Forecasting, Admission Control, and Capacity Explanation Cards

Miranda Bai

Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023 · 30.07.2026 · 10 (01) · 25-38

DOI: 10.60087/jaigs.v10i01.489 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023, online erschienen am 30.07.2026. Verfasst von Miranda Bai. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language model services face seasonal but abrupt token demand, making point forecasts unsafe and uniform reserves wasteful. This study joins multi-horizon forecasting, calibrated uncertainty, capacity selection, admission, and operator explanations on 1,429,737 BurstGPT_1 records spanning 60.995 trace-relative days. Five compact models predict requests and tokens 15 minutes, one hour, and six hours ahead. Validation-selected token models attain test mean absolute errors of 64,716.70, 77,321.65, and 113,388.62 tokens. At one hour, a 90% conformal upper policy costs 3.223 times observed demand and violates capacity in 5.802% of intervals. Median and P90 response gates eliminate accepted-load violations while admitting 85.472% and 74.337% of requests. Capacity cards expose forecasts, buffers, baselines, realizations, and status. Explicit risk selection, rather than point accuracy alone, governs defensible capacity planning.

Zitieren

APA

Bai, M. (2026). Risk-Aware LLM Serving with Multi-Horizon Demand Forecasting, Admission Control, and Capacity Explanation Cards. Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023, 10(01), 25-38. https://doi.org/10.60087/jaigs.v10i01.489

Vancouver

Bai M. Risk-Aware LLM Serving with Multi-Horizon Demand Forecasting, Admission Control, and Capacity Explanation Cards. Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023. 2026;10(01):25-38. doi:10.60087/jaigs.v10i01.489

BibTeX
@article{Bai2026,
  author = {Bai, Miranda},
  title = {Risk-Aware LLM Serving with Multi-Horizon Demand Forecasting, Admission Control, and Capacity Explanation Cards},
  journal = {Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023},
  year = {2026},
  volume = {10},
  number = {01},
  pages = {25--38},
  doi = {10.60087/jaigs.v10i01.489}
}
Kostenlos starten →