From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks
The VLDB Journal · 2025-03 · 34 (2)
DOI: 10.1007/s00778-024-00897-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift The VLDB Journal, online erschienen am 31.01.2025. Verfasst von Jingshu Peng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 84 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Hong Kong University of Science and Technology.
Abstract (Originalsprache)
Graph neural networks (GNNs) have emerged due to their success at modeling graph data. Yet, it is challenging for GNNs to efficiently scale to large graphs. Thus, distributed GNNs come into play. To avoid communication caused by expensive data movement between workers, we propose S ancus and its advanced version S ancus "Equation missing", the staleness and quantization-aware communication-avoiding decentralized GNN system. By introducing a set of novel bounded embedding staleness metrics and adaptively skipping broadcasts, S ancus abstracts decentralized GNN processing as sequential matrix multiplication and uses historical embeddings via cache. To further mitigate the communication volume, S ancus "Equation missing" conducts quantization-aware communication on embeddings to reduce the size of broadcast messages. Theoretically, we show bounded approximation errors of embeddings and gradients with a known fastest convergence guarantee. Empirically, we evaluate S ancus and S ancus "Equation missing" with common GNN models via different system setups on large-scale benchmark datasets. Compared to SOTA works, S ancus "Equation missing" can avoid up to $$86\%$$ 86 % communication with $$3.0\times $$ 3.0 × faster throughput on average without accuracy loss.
Förderung
- Hong Kong University of Science and Technology
Zitieren
APA
Peng, J., Liu, Q., Chen, Z., Shao, Y., Shen, Y., Chen, L., & Cao, J. (2025). From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks. The VLDB Journal, 34(2). https://doi.org/10.1007/s00778-024-00897-2
Vancouver
Peng J, Liu Q, Chen Z, Shao Y, Shen Y, Chen L, et al. From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks. The VLDB Journal. 2025;34(2). doi:10.1007/s00778-024-00897-2
BibTeX
@article{Peng2025,
author = {Peng, Jingshu and Liu, Qiyu and Chen, Zhao and Shao, Yingxia and Shen, Yanyan and Chen, Lei and Cao, Jiannong},
title = {From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks},
journal = {The VLDB Journal},
year = {2025},
volume = {34},
number = {2},
doi = {10.1007/s00778-024-00897-2}
}
EUVIMED