The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey

J Vatter, R Mayer, HA Jacobsen - ACM Computing Surveys, 2023 - dl.acm.org
Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

Computing graph neural networks: A survey from algorithms to accelerators

S Abadal, A Jain, R Guirado, J López-Alonso… - ACM Computing …, 2021 - dl.acm.org
Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent
years owing to their capability to model and learn from graph-structured data. Such an ability …

Graph-less neural networks: Teaching old mlps new tricks via distillation

S Zhang, Y Liu, Y Sun, N Shah - arXiv preprint arXiv:2110.08727, 2021 - arxiv.org
Graph Neural Networks (GNNs) are popular for graph machine learning and have shown
great results on wide node classification tasks. Yet, they are less popular for practical …

Dorylus: Affordable, scalable, and accurate {GNN} training with distributed {CPU} servers and serverless threads

J Thorpe, Y Qiao, J Eyolfson, S Teng, G Hu… - … USENIX Symposium on …, 2021 - usenix.org
A graph neural network (GNN) enables deep learning on structured graph data. There are
two major GNN training obstacles: 1) it relies on high-end servers with many GPUs which …

P3: Distributed deep graph learning at scale

S Gandhi, AP Iyer - 15th {USENIX} Symposium on Operating Systems …, 2021 - usenix.org
Graph Neural Networks (GNNs) have gained significant attention in the recent past, and
become one of the fastest growing subareas in deep learning. While several new GNN …

Towards graph foundation models: A survey and beyond

J Liu, C Yang, Z Lu, J Chen, Y Li, M Zhang… - arXiv preprint arXiv …, 2023 - arxiv.org
Emerging as fundamental building blocks for diverse artificial intelligence applications,
foundation models have achieved notable success across natural language processing and …

ByteGNN: efficient graph neural network training at large scale

C Zheng, H Chen, Y Cheng, Z Song, Y Wu… - Proceedings of the …, 2022 - dl.acm.org
Graph neural networks (GNNs) have shown excellent performance in a wide range of
applications such as recommendation, risk control, and drug discovery. With the increase in …

GNNLab: a factored system for sample-based GNN training over GPUs

J Yang, D Tang, X Song, L Wang, Q Yin… - Proceedings of the …, 2022 - dl.acm.org
We propose GNNLab, a sample-based GNN training system in a single machine multi-GPU
setup. GNNLab adopts a factored design for multiple GPUs, where each GPU is dedicated to …

Sancus: staleness-aware communication-avoiding full-graph decentralized training in large-scale graph neural networks

J Peng, Z Chen, Y Shao, Y Shen, L Chen… - Proceedings of the VLDB …, 2022 - dl.acm.org
Graph neural networks (GNNs) have emerged due to their success at modeling graph data.
Yet, it is challenging for GNNs to efficiently scale to large graphs. Thus, distributed GNNs …

Linkless link prediction via relational distillation

Z Guo, W Shiao, S Zhang, Y Liu… - International …, 2023 - proceedings.mlr.press
Abstract Graph Neural Networks (GNNs) have shown exceptional performance in the task of
link prediction. Despite their effectiveness, the high latency brought by non-trivial …