EUVIMEDEUVIMED entdecken
Crossref

CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks

Dianxuan Fu, Yihao Zhang, Xiaomin Liu, Qizhi Qiu, Yicheng Xu, Xingyu Liu, Xin Miao, Ping Du, Shikui Shen, Weisheng Hu, Qunbi Zhuge

Journal of Optical Communications and Networking · 01.08.2026 · 18 (8) · 873

DOI: 10.1364/jocn.600581 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Optical Communications and Networking, online erschienen am 31.07.2026. Verfasst von Dianxuan Fu und 10 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Gefördert von: Shanghai Jiao Tong University, State Key Laboratory of Satellite Network and Shanghai Key Laboratory of Satellite Network.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The explosive development in artificial intelligence (AI) like large language models (LLMs) requires massive graphics processing units (GPUs) during training, making distributed machine learning (DML) across multiple data centers (DCs) a practical approach to obtaining sufficient training resources. Among cross-DC DML techniques, pipeline parallelism (PP) is a natural solution, which partitions LLM layers across GPUs and orchestrates point-to-point communication between adjacent nodes. However, the high communication latency induces severe GPU idle time (“bubble”) due to poor communication and computation time overlapping in inter-DC PP. While prior works have explored partitioning and reordering computation tasks, the resource allocation for inter-DC communication between pipelined computing tasks over optical networks has been overlooked, resulting in higher GPU idle time and request congestion. Therefore, it is extremely important and challenging to propose an advanced inter-DC network resource allocation solution under the complex PP workload. In this work, we propose a cross-domain communication-bound-aware (CD-CBA) PP scheduling framework for multi-DC optical networks. Simulation results show that the proposed framework lowers the LLM training iteration time by up to 31.25%, decreases the bubble ratio by up to 11.81%, and reduces the transmission request blocking by up to 19.33%.

Förderung

  • Shanghai Jiao Tong University
  • State Key Laboratory of Satellite Network and Shanghai Key Laboratory of Satellite Network

Zitieren

APA

Fu, D., Zhang, Y., Liu, X., Qiu, Q., Xu, Y., Liu, X., Miao, X., Du, P., Shen, S., Hu, W., & Zhuge, Q. (2026). CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks. Journal of Optical Communications and Networking, 18(8), 873. https://doi.org/10.1364/jocn.600581

Vancouver

Fu D, Zhang Y, Liu X, Qiu Q, Xu Y, Liu X, et al. CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks. Journal of Optical Communications and Networking. 2026;18(8):873. doi:10.1364/jocn.600581

BibTeX
@article{Fu2026,
  author = {Fu, Dianxuan and Zhang, Yihao and Liu, Xiaomin and Qiu, Qizhi and Xu, Yicheng and Liu, Xingyu and Miao, Xin and Du, Ping and Shen, Shikui and Hu, Weisheng and Zhuge, Qunbi},
  title = {CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks},
  journal = {Journal of Optical Communications and Networking},
  year = {2026},
  volume = {18},
  number = {8},
  pages = {873},
  doi = {10.1364/jocn.600581}
}
Kostenlos starten →