Overparameterized neural networks can be highly accurate on average on an iid test set yet
consistently fail on atypical groups of the data (eg, by learning spurious correlations that …

A central goal of machine learning is to learn robust representations that capture the
fundamental relationship between inputs and output labels. However, minimizing training …

We often see undesirable tradeoffs in robust machine learning where out-of-distribution
(OOD) accuracy is at odds with in-distribution (ID) accuracy. A robust classifier obtained via …

Empirical risk minimization (ERM) of neural networks is prone to over-reliance on spurious
correlations and poor generalization on minority groups. The recent deep feature …

Real-world machine learning problems often exhibit shifts between the source and target
distributions, in which source data does not fully convey the desired behavior on target …

Abstract Machine learning algorithms typically assume that training and test examples are
drawn from the same distribution. However, distribution shift is a common problem in real …

Standard neural networks struggle to generalize under distribution shifts in computer vision.
Fortunately, combining multiple networks can consistently improve out-of-distribution …

Neural networks trained with SGD were recently shown to rely preferentially on linearly-
predictive features and can ignore complex, equally-predictive ones. This simplicity bias can …

Generalization under distributional shift is an open challenge for machine learning. Invariant
Risk Minimization (IRM) is a promising framework to tackle this issue by extracting invariant …

While second order optimizers such as natural gradient descent (NGD) often speed up
optimization, their effect on generalization has been called into question. This work presents …