Neural networks have achieved remarkable empirical performance, while the current
theoretical analysis is not adequate for understanding their success, eg, the Neural Tangent …

Linear classifiers and leaky ReLU networks trained by gradient flow on the logistic loss have
an implicit bias towards solutions which satisfy the Karush–Kuhn–Tucker (KKT) conditions …

In this work, we study the implications of the implicit bias of gradient flow on generalization
and adversarial robustness in ReLU networks. We focus on a setting where the data …

Neural networks trained by gradient descent (GD) have exhibited a number of surprising
generalization behaviors. First, they can achieve a perfect fit to noisy training data and still …

Transformers have demonstrated great power in the recent development of large
foundational models. In particular, the Vision Transformer (ViT) has brought revolutionary …

The phenomenon of benign overfitting, where a trained neural network perfectly fits noisy
training data but still achieves near-optimal test performance, has been extensively studied …

We study the least-square regression problem with a two-layer fully-connected neural
network, with ReLU activation function, trained by gradient flow. Our first result is a …

Modern over-parameterized neural networks can be trained to fit the training data perfectly
while still maintaining a high generalization performance. This" benign overfitting" …

Access to clean water is a fundamental human need, yet millions of people worldwide still
lack access to safe drinking water. Traditional water quality assessments, though reliable …

In this note, we study how neural networks with a single hidden layer and ReLU activation
interpolate data drawn from a radially symmetric distribution with target labels 1 at the origin …