We argue that the optimization plays a crucial role in generalization of deep learning models
through implicit regularization. We do this by demonstrating that generalization ability is not …

Characterizing how neural network depth, width, and dataset size jointly impact model
quality is a central problem in deep learning theory. We give here a complete solution in the …

Why does training deep neural networks using stochastic gradient descent (SGD) result in a
generalization error that does not worsen with the number of parameters in the network? To …

Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
Page 1 Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two …

Neural networks trained by gradient descent (GD) have exhibited a number of surprising
generalization behaviors. First, they can achieve a perfect fit to noisy training data and still …

Graph Neural Networks (GNNs) have achieved a lot of success on graph-structured data.
However, it is observed that the performance of graph neural networks does not improve as …

The implicit biases of gradient-based optimization algorithms are conjectured to be a major
factor in the success of modern deep learning. In this work, we investigate the implicit bias of …

In classical machine learning, regression is treated as a black box process of identifying a
suitable function from a hypothesis set without attempting to gain insight into the mechanism …

Overparameterized neural networks (NNs) are observed to generalize well even when
trained to perfectly fit noisy data. This phenomenon motivated a large body of work on" …

Despite extensive studies, the underlying reason as to why overparameterizedneural
networks can generalize remains elusive. Existing theory shows that common stochastic …