We study how neural networks trained by gradient descent extrapolate, ie, what they learn
outside the support of the training distribution. Previous works report mixed empirical results …

In recent years, several results in the supervised learning setting suggested that classical
statistical learning-theoretic measures, such as VC dimension, do not adequately explain …

Abstract Graph Neural Networks (GNNs) have been studied through the lens of expressive
power and generalization. However, their optimization properties are less well understood …

Since the proposal of the graph neural network (GNN) by Gori et al.(2005) and Scarselli et
al.(2008), one of the major problems in training GNNs was their struggle to propagate …

For certain infinitely-wide neural networks, the neural tangent kernel (NTK) theory fully
characterizes generalization, but for the networks used in practice, the empirical NTK only …

Graph neural networks (GNNs) are widely used in machine learning for graph-structured
data. Even though GNNs have achieved remarkable success in real-world applications …

Neural networks that process the parameters of other neural networks find applications in
domains as diverse as classifying implicit neural representations, generating neural network …

Oversmoothing is a central challenge of building more powerful Graph Neural Networks
(GNNs). While previous works have only demonstrated that oversmoothing is inevitable …

Graph neural networks (GNNs), as the de-facto model class for representation learning on
graphs, are built upon the multi-layer perceptrons (MLP) architecture with additional …

Despite the recent success of graph neural networks (GNN), common architectures often
exhibit significant limitations, including sensitivity to oversmoothing, long-range …