MixDNN: Scheduling Optimization of Distributed DNN Training in Heterogeneous GPU Environments
Electronics · 2026-11-11 · 15 (22) · 5236
DOI: 10.3390/electronics15225236 ↗At a glance
Journal article in Electronics, published online on 2026-11-11. Written by Zhang Xiao and 3 co-authors. 31 references. The full text is freely available (open access, CC BY 4.0). Funded by: Fundamental Research Funds for the Central Universities, National Natural Science Foundation of China.
Abstract (original language)
With the rapid advancement of deep neural networks (DNN), the scale of workloads continues to expand, exhibiting diverse training characteristics. Traditional scheduling frameworks for PyTorch and TensorFlow, such as Kubernetes, typically assume uniform training workloads across samples within a given task. Consequently, existing frameworks adopt static parallelism strategies throughout the entire training process, leading to suboptimal performance in heterogeneous distributed training environments. As model complexity and GPU heterogeneity grow, this rigidity results in inefficient resource utilization and degraded scalability. To address these challenges, we propose MixDNN, an event-driven rolling-horizon scheduler for distributed DNN models in heterogeneous GPU environments. First, MixDNN divides submitted jobs into two categories, hard-model and soft-model. Then, MixDNN associates utility with different resource requests to represent GPU computing performance. At each job-arrival or job-completion event, MixDNN selects execution configurations and planned start times for waiting jobs by maximizing overall utility subject to GPU-capacity constraints. Running jobs are not preempted, migrated, or reconfigured. In the evaluation, MixDNN reduces parallel training time by up to 11.45% compared to Kubernetes and up to 70% compared to static scheduling, especially for distributed DNN models in heterogeneous GPU environments.
Funding
- Fundamental Research Funds for the Central Universities
- National Natural Science Foundation of China
Cite
APA
Xiao, Z., Pengwei, Y., Xuwen, L., & Yumeng, W. (2026). MixDNN: Scheduling Optimization of Distributed DNN Training in Heterogeneous GPU Environments. Electronics, 15(22), 5236. https://doi.org/10.3390/electronics15225236
Vancouver
Xiao Z, Pengwei Y, Xuwen L, Yumeng W. MixDNN: Scheduling Optimization of Distributed DNN Training in Heterogeneous GPU Environments. Electronics. 2026;15(22):5236. doi:10.3390/electronics15225236
BibTeX
@article{Xiao2026,
author = {Xiao, Zhang and Pengwei, Ye and Xuwen, Luo and Yumeng, Wang},
title = {MixDNN: Scheduling Optimization of Distributed DNN Training in Heterogeneous GPU Environments},
journal = {Electronics},
year = {2026},
volume = {15},
number = {22},
pages = {5236},
doi = {10.3390/electronics15225236}
}
EUVIMED