Optimization of graph neural networks: Implicit acceleration by skip connections and more depth

K Xu, M Zhang, S Jegelka… - … on Machine Learning, 2021 - proceedings.mlr.press
Abstract Graph Neural Networks (GNNs) have been studied through the lens of expressive
power and generalization. However, their optimization properties are less well understood …

A survey on the expressive power of graph neural networks

R Sato - arXiv preprint arXiv:2003.04078, 2020 - arxiv.org
Graph neural networks (GNNs) are effective machine learning models for various graph
learning problems. Despite their empirical successes, the theoretical limitations of GNNs …

On the bottleneck of graph neural networks and its practical implications

U Alon, E Yahav - arXiv preprint arXiv:2006.05205, 2020 - arxiv.org
Since the proposal of the graph neural network (GNN) by Gori et al.(2005) and Scarselli et
al.(2008), one of the major problems in training GNNs was their struggle to propagate …

Towards understanding generalization of graph neural networks

H Tang, Y Liu - International Conference on Machine …, 2023 - proceedings.mlr.press
Graph neural networks (GNNs) are widely used in machine learning for graph-structured
data. Even though GNNs have achieved remarkable success in real-world applications …

Improving graph neural networks with learnable propagation operators

M Eliasof, L Ruthotto, E Treister - … Conference on Machine …, 2023 - proceedings.mlr.press
Abstract Graph Neural Networks (GNNs) are limited in their propagation operators. In many
cases, these operators often contain non-negative elements only and are shared across …

Mlpinit: Embarrassingly simple gnn training acceleration with mlp initialization

X Han, T Zhao, Y Liu, X Hu, N Shah - arXiv preprint arXiv:2210.00102, 2022 - arxiv.org
Training graph neural networks (GNNs) on large graphs is complex and extremely time
consuming. This is attributed to overheads caused by sparse matrix multiplication, which are …

Are graph convolutional networks with random weights feasible?

C Huang, M Li, F Cao, H Fujita, Z Li… - IEEE Transactions on …, 2022 - ieeexplore.ieee.org
Graph Convolutional Networks (GCNs), as a prominent example of graph neural networks,
are receiving extensive attention for their powerful capability in learning node …

Training graph neural networks with 1000 layers

G Li, M Müller, B Ghanem… - … conference on machine …, 2021 - proceedings.mlr.press
Deep graph neural networks (GNNs) have achieved excellent results on various tasks on
increasingly large graph datasets with millions of nodes and edges. However, memory …

Graph neural networks are inherently good generalizers: Insights by bridging gnns and mlps

C Yang, Q Wu, J Wang, J Yan - arXiv preprint arXiv:2212.09034, 2022 - arxiv.org
Graph neural networks (GNNs), as the de-facto model class for representation learning on
graphs, are built upon the multi-layer perceptrons (MLP) architecture with additional …

Graph neural networks: Methods, applications, and opportunities

L Waikhom, R Patgiri - arXiv preprint arXiv:2108.10733, 2021 - arxiv.org
In the last decade or so, we have witnessed deep learning reinvigorating the machine
learning field. It has solved many problems in the domains of computer vision, speech …