EUVIMEDEUVIMED entdecken
Crossref

SMore: Enhancing GPU Utilization in Deep Learning Clusters by Serverless-Based Co-Location Scheduling

Junhan Liu, Zinuo Cai, Yumou Liu, Hao Li, Zongpu Zhang, Ruhui Ma, Rajkumar Buyya

IEEE Transactions on Parallel and Distributed Systems · 2025-05 · 36 (5) · 903-917

DOI: 10.1109/tpds.2025.3548320 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Parallel and Distributed Systems, 2025-05. Verfasst von Junhan Liu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 54 Literaturangaben. Gefördert von: Shanghai Key Laboratory of Scalable Computing and Systems.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep learning (DL) clusters allow machine learning practitioners to submit their computation-intensive tasks, with GPUs accelerating their execution process. However, GPUs in current deep learning clusters are often under-utilized, which hampers the job performance and overall cluster throughput. It is urgent to improve GPU utilization, but existing works lack research on fine-grained allocation for GPU resources, as it typically allocates GPUs as indivisible units. Serverless computing reveals an opportunity to optimize utilization with fine-grained resource allocation methods, but it requires addressing three main challenges: co-location performance degradation, service level objectives guarantee of serverless functions, and cold start overhead. We propose SMore, a framework based on serverless computing to optimize GPU resource utilization of DL clusters. SMore dynamically predicts the possible co-location performance degradation and leverages a degradation-aware scheduling algorithm to ensure that the co-location decisions do not impact workload performance. It also dynamically preloads or offloads DL models by predicting the request numbers of the subsequent period to address the cold start issue. Through actual trace testing on the prototype of SMore, we find that the average GPU utilization can be increased by 34% with degradation being controlled effectively.

Förderung

  • Shanghai Key Laboratory of Scalable Computing and Systems

Zitieren

APA

Liu, J., Cai, Z., Liu, Y., Li, H., Zhang, Z., Ma, R., & Buyya, R. (2025). SMore: Enhancing GPU Utilization in Deep Learning Clusters by Serverless-Based Co-Location Scheduling. IEEE Transactions on Parallel and Distributed Systems, 36(5), 903-917. https://doi.org/10.1109/tpds.2025.3548320

Vancouver

Liu J, Cai Z, Liu Y, Li H, Zhang Z, Ma R, et al. SMore: Enhancing GPU Utilization in Deep Learning Clusters by Serverless-Based Co-Location Scheduling. IEEE Transactions on Parallel and Distributed Systems. 2025;36(5):903-917. doi:10.1109/tpds.2025.3548320

BibTeX
@article{Liu2025,
  author = {Liu, Junhan and Cai, Zinuo and Liu, Yumou and Li, Hao and Zhang, Zongpu and Ma, Ruhui and Buyya, Rajkumar},
  title = {SMore: Enhancing GPU Utilization in Deep Learning Clusters by Serverless-Based Co-Location Scheduling},
  journal = {IEEE Transactions on Parallel and Distributed Systems},
  year = {2025},
  volume = {36},
  number = {5},
  pages = {903--917},
  doi = {10.1109/tpds.2025.3548320}
}
Kostenlos starten →