Modern machine learning models often employ a huge number of parameters and are
typically optimized to have zero training loss; yet surprisingly, they possess near-optimal …

State-of-the-art Graph Neural Networks (GNNs) have limited scalability with respect to the
graph and model sizes. On large graphs, increasing the model depth often means …

Empirical evidence suggests that neural networks with ReLU activations generalize better
with over-parameterization. However, there is currently no theoretical analysis that explains …

Several recent works use positional encodings to extend the receptive fields of graph neural
network (GNN) layers equipped with attention mechanisms. These techniques, however …

Understanding generalization and robustness of machine learning models fundamentally
relies on assuming an appropriate metric on the data space. Identifying such a metric is …

In the past few years, graph neural networks (GNNs) have become the de facto model of
choice for graph classification. While, from the theoretical viewpoint, most GNNs can operate …

Graph neural networks (GNNs) have become a popular approach to integrating structural
inductive biases into NLP models. However, there has been little work on interpreting them …

Abstract Graph Convolutional Networks (GCNs) are known to suffer from performance
degradation as the number of layers increases, which is usually attributed to over …

Graph Neural Network (GNN) research has concentrated on improving convolutional layers,
with little attention paid to developing graph pooling layers. Yet pooling layers can enable …

Normalization is known to help the optimization of deep neural networks. Curiously, different
architectures require specialized normalization methods. In this paper, we study what …