EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS

PengFei Li, EnMiao Feng, GaiKe Wang, TianJun Mo

Innovation and Technology Studies · 04.06.2024 · 1 (1) · 48-56

DOI: 10.61784/its4028 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Innovation and Technology Studies, online erschienen am 04.06.2024. Verfasst von PengFei Li und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

AI computing clusters increasingly combine several generations of GPUs with CPU-only servers and host a mix of long-running training jobs and short, latency-sensitive inference or interactive jobs whose demands span GPUs, CPU cores and memory. Scheduling such clusters is a sequential, multi-objective combinatorial problem: heuristic and myopic optimization-based schedulers ignore how current placements constrain future arrivals, whereas pure deep reinforcement learning (DRL) schedulers face an exploding action space and offer no feasibility guarantee. This paper presents HyRIS, a hybrid scheduler that couples DRL with mixed-integer programming (MIP). At each decision epoch, a permutation-equivariant Transformer policy trained by proximal policy optimization emits state-dependent weights of the job-utility function and per-accelerator shadow prices, and a compact MIP converts them into a gang-, multi-resource- and heterogeneity-aware placement that satisfies all capacity constraints by construction. Zero-initialized residual heads anchor the policy to a hand-tuned MIP so that learning starts from a strong and safe operating point, and a reward derived from Little’s law jointly accounts for job completion time, monetary cost and service-level-objective (SLO) violations. Trace-driven experiments on the Alibaba PAI GPU cluster trace with V100, P100, T4, legacy GPUs and CPU-only nodes show that, under heavy load, HyRIS reduces the composite scheduling objective by 13.0–23.9% relative to FIFO, fragmentation-aware, multi-resource packing, heterogeneity-aware LP and pure DRL schedulers and remains within 1.9% of the best baseline (Static-MIP), while keeping the SLO-violation rate of latency-sensitive jobs at 11.9% (shortest-job-first: 10.8%). Each MIP is solved in 2.6 ms on average, making HyRIS practical for online use.

Zitieren

APA

Li, P., Feng, E., Wang, G., & Mo, T. (2024). HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS. Innovation and Technology Studies, 1(1), 48-56. https://doi.org/10.61784/its4028

Vancouver

Li P, Feng E, Wang G, Mo T. HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS. Innovation and Technology Studies. 2024;1(1):48-56. doi:10.61784/its4028

BibTeX
@article{Li2024,
  author = {Li, PengFei and Feng, EnMiao and Wang, GaiKe and Mo, TianJun},
  title = {HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS},
  journal = {Innovation and Technology Studies},
  year = {2024},
  volume = {1},
  number = {1},
  pages = {48--56},
  doi = {10.61784/its4028}
}
Kostenlos starten →