EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks

Jingshu Peng, Qiyu Liu, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, Jiannong Cao

The VLDB Journal · 2025-03 · 34 (2)

DOI: 10.1007/s00778-024-00897-2 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The VLDB Journal, online erschienen am 31.01.2025. Verfasst von Jingshu Peng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 84 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Hong Kong University of Science and Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Graph neural networks (GNNs) have emerged due to their success at modeling graph data. Yet, it is challenging for GNNs to efficiently scale to large graphs. Thus, distributed GNNs come into play. To avoid communication caused by expensive data movement between workers, we propose S ancus and its advanced version S ancus "Equation missing", the staleness and quantization-aware communication-avoiding decentralized GNN system. By introducing a set of novel bounded embedding staleness metrics and adaptively skipping broadcasts, S ancus abstracts decentralized GNN processing as sequential matrix multiplication and uses historical embeddings via cache. To further mitigate the communication volume, S ancus "Equation missing" conducts quantization-aware communication on embeddings to reduce the size of broadcast messages. Theoretically, we show bounded approximation errors of embeddings and gradients with a known fastest convergence guarantee. Empirically, we evaluate S ancus and S ancus "Equation missing" with common GNN models via different system setups on large-scale benchmark datasets. Compared to SOTA works, S ancus "Equation missing" can avoid up to $$86\%$$ 86 % communication with $$3.0\times $$ 3.0 × faster throughput on average without accuracy loss.

Förderung

  • Hong Kong University of Science and Technology

Zitieren

APA

Peng, J., Liu, Q., Chen, Z., Shao, Y., Shen, Y., Chen, L., & Cao, J. (2025). From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks. The VLDB Journal, 34(2). https://doi.org/10.1007/s00778-024-00897-2

Vancouver

Peng J, Liu Q, Chen Z, Shao Y, Shen Y, Chen L, et al. From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks. The VLDB Journal. 2025;34(2). doi:10.1007/s00778-024-00897-2

BibTeX
@article{Peng2025,
  author = {Peng, Jingshu and Liu, Qiyu and Chen, Zhao and Shao, Yingxia and Shen, Yanyan and Chen, Lei and Cao, Jiannong},
  title = {From Sancus to Sancus $$^q$$: staleness and quantization-aware full-graph decentralized training in graph neural networks},
  journal = {The VLDB Journal},
  year = {2025},
  volume = {34},
  number = {2},
  doi = {10.1007/s00778-024-00897-2}
}
Kostenlos starten →