Distributed Graph Neural Network Training: A Survey
ACM Computing Surveys · 31.08.2024 · 56 (8) · 1-39
DOI: 10.1145/3648358 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 10.04.2024. Verfasst von Yingxia Shao und 8 weiteren Autor:innen. Die Arbeit wurde bisher 100-mal zitiert (Crossref) und nennt 174 Literaturangaben. Gefördert von: National Science and Technology Major Project, National Natural Science Foundation of China, Beijing Nova Program.
Abstract (Originalsprache)
Graph neural networks (GNNs) are a type of deep learning models that are trained on graphs and have been successfully applied in various domains. Despite the effectiveness of GNNs, it is still challenging for GNNs to efficiently scale to large graphs. As a remedy, distributed computing becomes a promising solution of training large-scale GNNs, since it is able to provide abundant computing resources. However, the dependency of graph structure increases the difficulty of achieving high-efficiency distributed GNN training, which suffers from the massive communication and workload imbalance. In recent years, many efforts have been made on distributed GNN training, and an array of training algorithms and systems have been proposed. Yet, there is a lack of systematic review of the optimization techniques for the distributed execution of GNN training. In this survey, we analyze three major challenges in distributed GNN training: massive feature communication, the loss of model accuracy, and workload imbalance. Then, we introduce a new taxonomy for the optimization techniques in distributed GNN training that address the above challenges. The new taxonomy classifies existing techniques into four categories: GNN data partition, GNN batch generation, GNN execution model, and GNN communication protocol. We carefully discuss the techniques in each category. In the conclusion, we summarize existing distributed GNN systems for multi–graphics processing units (GPUs), GPU-clusters and central processing unit (CPU)-clusters, respectively, and present a discussion about the future direction of distributed GNN training.
Förderung
- National Science and Technology Major Project
- National Natural Science Foundation of China
- Beijing Nova Program
- Xiaomi Young Talents Program
- National Science Foundation of China
- Hong Kong RGC GRF Project
- CRF Project
- AOE Project
Zitieren
APA
Shao, Y., Li, H., Gu, X., Yin, H., Li, Y., Miao, X., Zhang, W., Cui, B., & Chen, L. (2024). Distributed Graph Neural Network Training: A Survey. ACM Computing Surveys, 56(8), 1-39. https://doi.org/10.1145/3648358
Vancouver
Shao Y, Li H, Gu X, Yin H, Li Y, Miao X, et al. Distributed Graph Neural Network Training: A Survey. ACM Computing Surveys. 2024;56(8):1-39. doi:10.1145/3648358
BibTeX
@article{Shao2024,
author = {Shao, Yingxia and Li, Hongzheng and Gu, Xizhi and Yin, Hongbo and Li, Yawen and Miao, Xupeng and Zhang, Wentao and Cui, Bin and Chen, Lei},
title = {Distributed Graph Neural Network Training: A Survey},
journal = {ACM Computing Surveys},
year = {2024},
volume = {56},
number = {8},
pages = {1--39},
doi = {10.1145/3648358}
}
EUVIMED