EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

LLM-Augmented Salable GPU Supply Forecasting for Disaggregated Recommendation Serving: Predicting Instance Readiness, Scheduling Delay, and Capacity Risk

Haowei Tu, Siming Zhao, Samuel He

Journal of Advanced Computing Systems · 24.09.2024 · 4 (9) · 85-102

DOI: 10.69987/jacs.2024.40908 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Advanced Computing Systems, online erschienen am 24.09.2024. Verfasst von Haowei Tu und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper presents a reproducible empirical study of salable GPU supply forecasting for disaggregated recommendation serving using the public Alibaba cluster-trace-gpu-v2025 dataset. The trace contains 23,871 latency-sensitive inference instances from 156 services, including 16,485 CPU-node instances and 7,386 heterogeneous GPU-node instances. We defined instance readiness as scheduled_time minus creation_time, defined an HN active-instance count as the salable GPU supply proxy, and created a grounded risk-memo layer that converts model outputs into capacity actions. The study evaluated three operational tasks on the downloaded CSV trace: scheduling-delay regression, 24-hour active-HN supply forecasting, and anomaly alerting for delayed scheduling, abnormal lifetime, and unusual resource requests. Chronological splits were used throughout to avoid future leakage. For scheduling delay, a random forest achieved the lowest validation MAE and a test MAE of 311.67 seconds; the RMSE remained high at 8,228.67 seconds because of rare extreme delays above two days. For 24-hour HN supply forecasting, a carry-forward temporal baseline achieved the best test MAE of 0.180 active instances and a SMAPE of 2.776%, showing that this serving trace is highly stable over six-hour windows. For anomaly alerting, a random forest classifier trained on train-derived silver labels reached test precision 0.982, recall 0.964, and F1 0.973. The LLM-style risk memo was generated only from structured output facts, and a programmatic consistency checker verified 11 numeric claims and 6 source citations with a hallucination rate of 0.000. All reported results, figures, and tables were produced by the accompanying code package; no unmeasured results were used.

Zitieren

APA

Tu, H., Zhao, S., & He, S. (2024). LLM-Augmented Salable GPU Supply Forecasting for Disaggregated Recommendation Serving: Predicting Instance Readiness, Scheduling Delay, and Capacity Risk. Journal of Advanced Computing Systems, 4(9), 85-102. https://doi.org/10.69987/jacs.2024.40908

Vancouver

Tu H, Zhao S, He S. LLM-Augmented Salable GPU Supply Forecasting for Disaggregated Recommendation Serving: Predicting Instance Readiness, Scheduling Delay, and Capacity Risk. Journal of Advanced Computing Systems. 2024;4(9):85-102. doi:10.69987/jacs.2024.40908

BibTeX
@article{Tu2024,
  author = {Tu, Haowei and Zhao, Siming and He, Samuel},
  title = {LLM-Augmented Salable GPU Supply Forecasting for Disaggregated Recommendation Serving: Predicting Instance Readiness, Scheduling Delay, and Capacity Risk},
  journal = {Journal of Advanced Computing Systems},
  year = {2024},
  volume = {4},
  number = {9},
  pages = {85--102},
  doi = {10.69987/jacs.2024.40908}
}
Kostenlos starten →