Scalable and Efficient Full-Graph GNN Training for Large Graphs
Proceedings of the ACM on Management of Data · 13.06.2023 · 1 (2) · 1-23
DOI: 10.1145/3589288 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 20.06.2023. Verfasst von Xinchen Wan und 5 weiteren Autor:innen. Die Arbeit wurde bisher 56-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Natural Science Foundation of China, National Natural Science Fund for the Excellent Young Scientists Fund Program, Hong Kong RGC.
Abstract (Originalsprache)
Graph Neural Networks (GNNs) have emerged as powerful tools to capture structural information from graph-structured data, achieving state-of-the-art performance on applications such as recommendation, knowledge graph, and search. Graphs in these domains typically contain hundreds of millions of nodes and billions of edges. However, previous GNN systems demonstrate poor scalability because large and interleaved computation dependencies in GNN training cause significant overhead in current parallelization methods. We present G3, a distributed system that can efficiently train GNNs over billion-edge graphs at scale. G3 introduces GNN hybrid parallelism which synthesizes three dimensions of parallelism to scale out GNN training by sharing intermediate results peer-to-peer in fine granularity, eliminating layer-wise barriers for global collective communication or neighbor replications as seen in prior works. G3 leverages locality-aware iterative partitioning and multi-level pipeline scheduling to exploit acceleration opportunities by distributing balanced workload among workers and overlapping computation with communication in both inter-layer and intra-layer training processes. We show via a prototype implementation and comprehensive experiments that G3 can achieve as much as 2.24x speedup in a 16-node cluster, and better final accuracy over prior works.
Förderung
- National Natural Science Foundation of China
- National Natural Science Fund for the Excellent Young Scientists Fund Program
- Hong Kong RGC
Zitieren
APA
Wan, X., Xu, K., Liao, X., Jin, Y., Chen, K., & Jin, X. (2023). Scalable and Efficient Full-Graph GNN Training for Large Graphs. Proceedings of the ACM on Management of Data, 1(2), 1-23. https://doi.org/10.1145/3589288
Vancouver
Wan X, Xu K, Liao X, Jin Y, Chen K, Jin X. Scalable and Efficient Full-Graph GNN Training for Large Graphs. Proceedings of the ACM on Management of Data. 2023;1(2):1-23. doi:10.1145/3589288
BibTeX
@article{Wan2023,
author = {Wan, Xinchen and Xu, Kaiqiang and Liao, Xudong and Jin, Yilun and Chen, Kai and Jin, Xin},
title = {Scalable and Efficient Full-Graph GNN Training for Large Graphs},
journal = {Proceedings of the ACM on Management of Data},
year = {2023},
volume = {1},
number = {2},
pages = {1--23},
doi = {10.1145/3589288}
}
EUVIMED