EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Entropy-Guided Hierarchical Scheduling for Elastic Distributed Deep Learning

Teh-Jen Sun, Eui-Nam Huh

Applied Sciences · 10.04.2026 · 16 (8) · 3725

DOI: 10.3390/app16083725 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 10.04.2026. Verfasst von Teh-Jen Sun und 1 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: MSIT (Ministry of Science and ICT), Korea, IITP, Global—Learning & Academic Research Institution.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Shared GPU clusters often execute multiple distributed training jobs concurrently under fluctuating contention. We reinterpret this setting as a two-scale control problem, where the micro scale captures intra-job learning dynamics and the macro scale captures inter-job resource arbitration. We propose an entropy-guided hierarchical framework that links these two scales through a unified uncertainty signal computed from training logits. Unlike existing uncertainty-aware methods that typically use uncertainty for only a single level of decision making, our approach uses the same entropy-based signal to jointly support both intra-job adaptation and inter-job scheduling within a hierarchical control loop. At the micro level, each worker estimates predictive uncertainty via normalized entropy and converts it into stable weights that drive epoch-level controls for uncertainty-aware data sharding, fixed-budget batch-size reallocation, and learning-rate modulation, while remaining compatible with standard synchronous data-parallel training. At the macro level, the same signal is aggregated into a job utility score that guides admission, ordering, and GPU quota assignment under contention. In large-scale workload-driven simulation, our method reduces average job completion time (JCT) by 23.7% and shortens cluster makespan by 15.7% relative to a strong learning-unaware baseline, demonstrating that uncertainty-aligned scheduling can improve cluster-level efficiency while preserving training correctness. We further validate scalability using a calibrated simulator up to 1024 nodes.

Förderung

  • MSIT (Ministry of Science and ICT), Korea
  • IITP
  • Global—Learning & Academic Research Institution
  • National Research Foundation of Korea

Zitieren

APA

Sun, T., & Huh, E. (2026). Entropy-Guided Hierarchical Scheduling for Elastic Distributed Deep Learning. Applied Sciences, 16(8), 3725. https://doi.org/10.3390/app16083725

Vancouver

Sun T, Huh E. Entropy-Guided Hierarchical Scheduling for Elastic Distributed Deep Learning. Applied Sciences. 2026;16(8):3725. doi:10.3390/app16083725

BibTeX
@article{Sun2026,
  author = {Sun, Teh-Jen and Huh, Eui-Nam},
  title = {Entropy-Guided Hierarchical Scheduling for Elastic Distributed Deep Learning},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {8},
  pages = {3725},
  doi = {10.3390/app16083725}
}
Kostenlos starten →