EUVIMEDEUVIMED entdecken
Crossref

Risk-Calibrated Multi-Resource Scheduling for Disaggregated AI Inference with LLM-Generated Evidence Memos

Justin Yang

Stout in Computer Science and Technology Studies · 25.01.2026 · 2 (1) · 24-46

DOI: 10.61424/6zd6va79 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Stout in Computer Science and Technology Studies, online erschienen am 25.01.2026. Verfasst von Justin Yang.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Disaggregated inference separates compute-intensive and accelerator-intensive stages, but it also turns capacity control into a coupled, multi-resource decision under time-varying demand. This study evaluates a risk-calibrated scheduling pipeline on the 2025 Alibaba trace of disaggregated deep-learning recommendation model services. The trace contains 23,871 instances from 156 long-running services over 31 days, including 16,485 compute-node instances and 7,386 heterogeneous-node instances. CPU, GPU, RDMA, memory, disk, deployment-density, and lifecycle fields were aggregated into 15-minute decision epochs. A chronological 60/20/20 design produced 1,689 training windows, 595 calibration windows, and 596 test observations; 595 held-out scheduling decisions were evaluated after lag initialization. The proposed RC-HGB-ILP policy combines a persistence-guarded histogram gradient-boosting forecast, a strictly online one-sided conformal demand envelope, and an integer node-mix planner. It was compared with reactive best-fit, a training-peak inventory, an exact one-step oracle, an uncalibrated HGB planner, and a density-constraint ablation. At 90% nominal coverage, mean empirical envelope coverage reached 94.03%. RC-HGB-ILP reduced the held-out scheduling-SLO violation rate from 18.98% for HGB-ILP to 5.08%, while increasing normalized operating cost by 979.55 cost units, or 0.098%. Relative to reactive best-fit, the violation rate fell from 9.32% to 5.08%. GPU utilization remained 90.25%, and the fragmentation index decreased to 0.31751. Paired epoch tests confirmed lower violation risk and fragmentation, with a small cost increase. An evidence-constrained memo protocol converted saved metrics into seven operational summaries; all 34 numeric claims matched their evidence records, all claims carried evidence identifiers, and all comparison directions were correct. The results show that calibrated upper-demand envelopes can absorb forecast drift at modest capacity cost while preserving high accelerator use, and that bounded evidence memos can expose the resulting trade-offs without numerical drift.

Zitieren

APA

Yang, J. (2026). Risk-Calibrated Multi-Resource Scheduling for Disaggregated AI Inference with LLM-Generated Evidence Memos. Stout in Computer Science and Technology Studies, 2(1), 24-46. https://doi.org/10.61424/6zd6va79

Vancouver

Yang J. Risk-Calibrated Multi-Resource Scheduling for Disaggregated AI Inference with LLM-Generated Evidence Memos. Stout in Computer Science and Technology Studies. 2026;2(1):24-46. doi:10.61424/6zd6va79

BibTeX
@article{Yang2026,
  author = {Yang, Justin},
  title = {Risk-Calibrated Multi-Resource Scheduling for Disaggregated AI Inference with LLM-Generated Evidence Memos},
  journal = {Stout in Computer Science and Technology Studies},
  year = {2026},
  volume = {2},
  number = {1},
  pages = {24--46},
  doi = {10.61424/6zd6va79}
}
Kostenlos starten →