Overparameterization improves robustness to covariate shift in high dimensions
N Tripuraneni, B Adlam… - Advances in Neural …, 2021 - proceedings.neurips.cc
A significant obstacle in the development of robust machine learning models is\emph
{covariate shift}, a form of distribution shift that occurs when the input distributions of the …
{covariate shift}, a form of distribution shift that occurs when the input distributions of the …
Uniform convergence of interpolators: Gaussian width, norm bounds and benign overfitting
We consider interpolation learning in high-dimensional linear regression with Gaussian
data, and prove a generic uniform convergence guarantee on the generalization error of …
data, and prove a generic uniform convergence guarantee on the generalization error of …
The implicit bias of benign overfitting
O Shamir - Conference on Learning Theory, 2022 - proceedings.mlr.press
The phenomenon of benign overfitting, where a predictor perfectly fits noisy training data
while attaining low expected loss, has received much attention in recent years, but still …
while attaining low expected loss, has received much attention in recent years, but still …
On linear stability of sgd and input-smoothness of neural networks
The multiplicative structure of parameters and input data in the first layer of neural networks
is explored to build connection between the landscape of the loss function with respect to …
is explored to build connection between the landscape of the loss function with respect to …
From tempered to benign overfitting in relu neural networks
G Kornowski, G Yehudai… - Advances in Neural …, 2024 - proceedings.neurips.cc
Overparameterized neural networks (NNs) are observed to generalize well even when
trained to perfectly fit noisy data. This phenomenon motivated a large body of work on" …
trained to perfectly fit noisy data. This phenomenon motivated a large body of work on" …
ResMem: Learn what you can and memorize the rest
The impressive generalization performance of modern neural networks is attributed in part to
their ability to implicitly memorize complex training patterns. Inspired by this, we explore a …
their ability to implicitly memorize complex training patterns. Inspired by this, we explore a …
Covariate shift in high-dimensional random feature regression
N Tripuraneni, B Adlam, J Pennington - arXiv preprint arXiv:2111.08234, 2021 - arxiv.org
A significant obstacle in the development of robust machine learning models is covariate
shift, a form of distribution shift that occurs when the input distributions of the training and test …
shift, a form of distribution shift that occurs when the input distributions of the training and test …
How do noise tails impact on deep ReLU networks?
How do noise tails impact on deep ReLU networks? Page 1 The Annals of Statistics 2024,
Vol. 52, No. 4, 1845–1871 https://doi.org/10.1214/24-AOS2428 © Institute of Mathematical …
Vol. 52, No. 4, 1845–1871 https://doi.org/10.1214/24-AOS2428 © Institute of Mathematical …
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks
In this paper, we investigate a two-layer fully connected neural network of the form f (X)= 1 d
1 a⊤ σ (WX), where X∈ d 0× n is a deterministic data matrix, W∈ R d 1× d 0 and a∈ R d 1 …
1 a⊤ σ (WX), where X∈ d 0× n is a deterministic data matrix, W∈ R d 1× d 0 and a∈ R d 1 …
Counterclr: Counterfactual contrastive learning with non-random missing data in recommendation
Recommender systems are designed to learn user preferences from observed feedback and
comprise many fundamental tasks, such as rating prediction and post-click conversion rate …
comprise many fundamental tasks, such as rating prediction and post-click conversion rate …