The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey

J Vatter, R Mayer, HA Jacobsen - ACM Computing Surveys, 2023 - dl.acm.org
Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

Edge-cloud polarization and collaboration: A comprehensive survey for ai

J Yao, S Zhang, Y Yao, F Wang, J Ma… - … on Knowledge and …, 2022 - ieeexplore.ieee.org
Influenced by the great success of deep learning via cloud computing and the rapid
development of edge chips, research in artificial intelligence (AI) has shifted to both of the …

A survey on green deep learning

J Xu, W Zhou, Z Fu, H Zhou, L Li - arXiv preprint arXiv:2111.05193, 2021 - arxiv.org
In recent years, larger and deeper models are springing up and continuously pushing state-
of-the-art (SOTA) results across various fields like natural language processing (NLP) and …

Distributed graph neural network training: A survey

Y Shao, H Li, X Gu, H Yin, Y Li, X Miao… - ACM Computing …, 2024 - dl.acm.org
Graph neural networks (GNNs) are a type of deep learning models that are trained on
graphs and have been successfully applied in various domains. Despite the effectiveness of …

EXACT: Scalable graph neural networks training via extreme activation compression

Z Liu, K Zhou, F Yang, L Li, R Chen… - … Conference on Learning …, 2021 - openreview.net
Training Graph Neural Networks (GNNs) on large graphs is a fundamental challenge due to
the high memory usage, which is mainly occupied by activations (eg, node embeddings) …

Large graph models: A perspective

Z Zhang, H Li, Z Zhang, Y Qin, X Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
Large models have emerged as the most recent groundbreaking achievements in artificial
intelligence, and particularly machine learning. However, when it comes to graphs, large …

QGTC: accelerating quantized graph neural networks via GPU tensor core

Y Wang, B Feng, Y Ding - Proceedings of the 27th ACM SIGPLAN …, 2022 - dl.acm.org
Over the most recent years, quantized graph neural network (QGNN) attracts lots of research
and industry attention due to its high robustness and low computation and memory …

Adaptive message quantization and parallelization for distributed full-graph gnn training

B Wan, J Zhao, C Wu - Proceedings of Machine Learning …, 2023 - proceedings.mlsys.org
Distributed full-graph training of Graph Neural Networks (GNNs) over large graphs is
bandwidth-demanding and time-consuming. Frequent exchanges of node features …

A survey on graph neural network acceleration: Algorithms, systems, and customized hardware

S Zhang, A Sohrabizadeh, C Wan, Z Huang… - arXiv preprint arXiv …, 2023 - arxiv.org
Graph neural networks (GNNs) are emerging for machine learning research on graph-
structured data. GNNs achieve state-of-the-art performance on many tasks, but they face …

VQ-GNN: A universal framework to scale up graph neural networks using vector quantization

M Ding, K Kong, J Li, C Zhu… - Advances in …, 2021 - proceedings.neurips.cc
Most state-of-the-art Graph Neural Networks (GNNs) can be defined as a form of graph
convolution which can be realized by message passing between direct neighbors or …