EUVIMEDEUVIMED entdecken
Crossref

Reliable and efficient RAR-based distributed model training in computing power network

Ling Chen, Yajie Li, Carlos Natalino, Yongcheng Li, Boxin Zhang, Yingbo Fan, Wei Wang, Yongli Zhao, Jie Zhang

Journal of Optical Communications and Networking · 01.05.2024 · 16 (5) · 527

DOI: 10.1364/jocn.511165 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Optical Communications and Networking, online erschienen am 22.04.2024. Verfasst von Ling Chen und 8 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 35 Literaturangaben. Gefördert von: Beijing Natural Science Foundation, Fundamental Research Funds for the Central Universities, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The computing power network (CPN) is a novel network technology that integrates computing power from the cloud, edge, and terminals using IP/optical cross-layer networks for distributed computing. CPNs can provide an effective solution for distributed model training (DMT). As a bandwidth optimization architecture based on data parallelism, ring all-reduce (RAR) is widely used in DMT. However, any node or link failure on the ring can interrupt or block the requests deployed on the ring. Meanwhile, due to the resource competition of batch RAR-based DMT requests, inappropriate scheduling strategies will also lead to low training efficiency or congestion. As far as we know, there is currently no research that considers the survivability of rings in scheduling strategies for RAR-based DMT. To fill this gap, we propose a scheduling scheme for RAR-based DMT requests in CPNs to optimize the allocation of computing and wavelength resources considering the time dimension while ensuring reliability. In practical scenarios, service providers may focus on different performance metrics. We formulate an integer linear programming (ILP) model and a RAR-based DMT deployment algorithm (RDDA) to solve this problem considering four optimization objectives under the premise of the minimum blocking rate: minimum computing resource consumption, minimum wavelength resource consumption, minimum training time, and maximum reliability. Simulation results demonstrate that our model satisfies the reliability requirements while achieving corresponding optimal performance for DMT requests under four optimization objectives.

Förderung

  • Beijing Natural Science Foundation
  • Fundamental Research Funds for the Central Universities
  • National Natural Science Foundation of China

Zitieren

APA

Chen, L., Li, Y., Natalino, C., Li, Y., Zhang, B., Fan, Y., Wang, W., Zhao, Y., & Zhang, J. (2024). Reliable and efficient RAR-based distributed model training in computing power network. Journal of Optical Communications and Networking, 16(5), 527. https://doi.org/10.1364/jocn.511165

Vancouver

Chen L, Li Y, Natalino C, Li Y, Zhang B, Fan Y, et al. Reliable and efficient RAR-based distributed model training in computing power network. Journal of Optical Communications and Networking. 2024;16(5):527. doi:10.1364/jocn.511165

BibTeX
@article{Chen2024,
  author = {Chen, Ling and Li, Yajie and Natalino, Carlos and Li, Yongcheng and Zhang, Boxin and Fan, Yingbo and Wang, Wei and Zhao, Yongli and Zhang, Jie},
  title = {Reliable and efficient RAR-based distributed model training in computing power network},
  journal = {Journal of Optical Communications and Networking},
  year = {2024},
  volume = {16},
  number = {5},
  pages = {527},
  doi = {10.1364/jocn.511165}
}
Kostenlos starten →