CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks
Journal of Optical Communications and Networking · 01.08.2026 · 18 (8) · 873
DOI: 10.1364/jocn.600581 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Optical Communications and Networking, online erschienen am 31.07.2026. Verfasst von Dianxuan Fu und 10 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Gefördert von: Shanghai Jiao Tong University, State Key Laboratory of Satellite Network and Shanghai Key Laboratory of Satellite Network.
Abstract (Originalsprache)
The explosive development in artificial intelligence (AI) like large language models (LLMs) requires massive graphics processing units (GPUs) during training, making distributed machine learning (DML) across multiple data centers (DCs) a practical approach to obtaining sufficient training resources. Among cross-DC DML techniques, pipeline parallelism (PP) is a natural solution, which partitions LLM layers across GPUs and orchestrates point-to-point communication between adjacent nodes. However, the high communication latency induces severe GPU idle time (“bubble”) due to poor communication and computation time overlapping in inter-DC PP. While prior works have explored partitioning and reordering computation tasks, the resource allocation for inter-DC communication between pipelined computing tasks over optical networks has been overlooked, resulting in higher GPU idle time and request congestion. Therefore, it is extremely important and challenging to propose an advanced inter-DC network resource allocation solution under the complex PP workload. In this work, we propose a cross-domain communication-bound-aware (CD-CBA) PP scheduling framework for multi-DC optical networks. Simulation results show that the proposed framework lowers the LLM training iteration time by up to 31.25%, decreases the bubble ratio by up to 11.81%, and reduces the transmission request blocking by up to 19.33%.
Förderung
- Shanghai Jiao Tong University
- State Key Laboratory of Satellite Network and Shanghai Key Laboratory of Satellite Network
Zitieren
APA
Fu, D., Zhang, Y., Liu, X., Qiu, Q., Xu, Y., Liu, X., Miao, X., Du, P., Shen, S., Hu, W., & Zhuge, Q. (2026). CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks. Journal of Optical Communications and Networking, 18(8), 873. https://doi.org/10.1364/jocn.600581
Vancouver
Fu D, Zhang Y, Liu X, Qiu Q, Xu Y, Liu X, et al. CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks. Journal of Optical Communications and Networking. 2026;18(8):873. doi:10.1364/jocn.600581
BibTeX
@article{Fu2026,
author = {Fu, Dianxuan and Zhang, Yihao and Liu, Xiaomin and Qiu, Qizhi and Xu, Yicheng and Liu, Xingyu and Miao, Xin and Du, Ping and Shen, Shikui and Hu, Weisheng and Zhuge, Qunbi},
title = {CD-CBA: a cross-domain communication-bound-aware resource allocation framework for pipeline-parallel distributed LLM training over dynamic multi-datacenter optical networks},
journal = {Journal of Optical Communications and Networking},
year = {2026},
volume = {18},
number = {8},
pages = {873},
doi = {10.1364/jocn.600581}
}
EUVIMED