Distributed graph neural network training: A survey

Y Shao, H Li, X Gu, H Yin, Y Li, X Miao… - ACM Computing …, 2024 - dl.acm.org
Graph neural networks (GNNs) are a type of deep learning models that are trained on
graphs and have been successfully applied in various domains. Despite the effectiveness of …

Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective

H Yuan, Y Liu, Y Zhang, X Ai, Q Wang, C Chen… - arXiv preprint arXiv …, 2023 - arxiv.org
Many Graph Neural Network (GNN) training systems have emerged recently to support
efficient GNN training. Since GNNs embody complex data dependencies between training …

Distributed Matrix-Based Sampling for Graph Neural Network Training

A Tripathy, K Yelick, A Buluc - Proceedings of Machine …, 2024 - proceedings.mlsys.org
Abstract Graph Neural Networks (GNNs) offer a compact and computationally efficient way
to learn embeddings and classifications on graph data. GNN models are frequently large …

Heta: Distributed Training of Heterogeneous Graph Neural Networks

Y Zhong, J Su, C Wu, M Wang - arXiv preprint arXiv:2408.09697, 2024 - arxiv.org
Heterogeneous Graph Neural Networks (HGNNs) leverage diverse semantic relationships
in Heterogeneous Graphs (HetGs) and have demonstrated remarkable learning …

MassiveGNN: Efficient Training via Prefetching for Massively Connected Distributed Graphs

A Sarkar, S Ghosh, NR Tallent… - 2024 IEEE International …, 2024 - ieeexplore.ieee.org
Graph Neural Networks (GNN) are indispensable in learning from graph-structured data, yet
their rising computational costs, especially on massively connected graphs, pose significant …

Distributed training of large graph neural networks with variable communication rates

J Cervino, MA Turja, H Mostafa, N Himayat… - arXiv preprint arXiv …, 2024 - arxiv.org
Training Graph Neural Networks (GNNs) on large graphs presents unique challenges due to
the large memory and computing requirements. Distributed GNN training, where the graph is …

Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch

S Bajaj, H Son, J Liu, H Guan, M Serafini - arXiv preprint arXiv:2406.00552, 2024 - arxiv.org
Graph Neural Networks (GNNs) have gained significant attention in recent years due to their
ability to learn representations of graph structured data. Two common methods for training …

GraNNDis: Fast Distributed Graph Neural Network Training Framework for Multi-Server Clusters

J Song, H Jang, H Lim, J Jung, Y Kim… - Proceedings of the 2024 …, 2024 - dl.acm.org
Graph neural networks (GNNs) are one of the rapidly growing fields within deep learning.
While many distributed GNN training frameworks have been proposed to increase the …

SuperGCN: General and Scalable Framework for GCN Training on CPU-powered Supercomputers

C Zhuang, P Chen, X Liu, R Yokota, N Dryden… - arXiv preprint arXiv …, 2024 - arxiv.org
Graph Convolutional Networks (GCNs) are widely used in various domains. However,
training distributed full-batch GCNs on large-scale graphs poses challenges due to …

NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism

X Ai, H Yuan, Z Ling, Q Wang, Y Zhang, Z Fu… - arXiv preprint arXiv …, 2024 - arxiv.org
Graph neural networks (GNNs) have emerged as a promising direction. Training large-scale
graphs that relies on distributed computing power poses new challenges. Existing …