HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS
Innovation and Technology Studies · 04.06.2024 · 1 (1) · 48-56
DOI: 10.61784/its4028 ↗Kurz gefasst
Fachartikel in der Zeitschrift Innovation and Technology Studies, online erschienen am 04.06.2024. Verfasst von PengFei Li und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
AI computing clusters increasingly combine several generations of GPUs with CPU-only servers and host a mix of long-running training jobs and short, latency-sensitive inference or interactive jobs whose demands span GPUs, CPU cores and memory. Scheduling such clusters is a sequential, multi-objective combinatorial problem: heuristic and myopic optimization-based schedulers ignore how current placements constrain future arrivals, whereas pure deep reinforcement learning (DRL) schedulers face an exploding action space and offer no feasibility guarantee. This paper presents HyRIS, a hybrid scheduler that couples DRL with mixed-integer programming (MIP). At each decision epoch, a permutation-equivariant Transformer policy trained by proximal policy optimization emits state-dependent weights of the job-utility function and per-accelerator shadow prices, and a compact MIP converts them into a gang-, multi-resource- and heterogeneity-aware placement that satisfies all capacity constraints by construction. Zero-initialized residual heads anchor the policy to a hand-tuned MIP so that learning starts from a strong and safe operating point, and a reward derived from Little’s law jointly accounts for job completion time, monetary cost and service-level-objective (SLO) violations. Trace-driven experiments on the Alibaba PAI GPU cluster trace with V100, P100, T4, legacy GPUs and CPU-only nodes show that, under heavy load, HyRIS reduces the composite scheduling objective by 13.0–23.9% relative to FIFO, fragmentation-aware, multi-resource packing, heterogeneity-aware LP and pure DRL schedulers and remains within 1.9% of the best baseline (Static-MIP), while keeping the SLO-violation rate of latency-sensitive jobs at 11.9% (shortest-job-first: 10.8%). Each MIP is solved in 2.6 ms on average, making HyRIS practical for online use.
Zitieren
APA
Li, P., Feng, E., Wang, G., & Mo, T. (2024). HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS. Innovation and Technology Studies, 1(1), 48-56. https://doi.org/10.61784/its4028
Vancouver
Li P, Feng E, Wang G, Mo T. HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS. Innovation and Technology Studies. 2024;1(1):48-56. doi:10.61784/its4028
BibTeX
@article{Li2024,
author = {Li, PengFei and Feng, EnMiao and Wang, GaiKe and Mo, TianJun},
title = {HYBRID DEEP REINFORCEMENT LEARNING AND INTEGER OPTIMIZATION FOR RESOURCE SCHEDULING IN HETEROGENEOUS AI COMPUTING CLUSTERS},
journal = {Innovation and Technology Studies},
year = {2024},
volume = {1},
number = {1},
pages = {48--56},
doi = {10.61784/its4028}
}
EUVIMED