Machine learning models often perform poorly on subgroups that are underrepresented in
the training data. Yet, little is understood on the variation in mechanisms that cause …

We develop a methodology for assessing the robustness of models to subpopulation shift---
specifically, their ability to generalize to novel data subpopulations that were not observed …

Subpopulation shift widely exists in many real-world machine learning applications, referring
to the training and test distributions containing the same subpopulation groups but varying in …

Robustness to distribution shifts is critical for deploying machine learning models in the real
world. Despite this necessity, there has been little work in defining the underlying …

Evaluating the performance of machine learning models on diverse and underrepresented
subgroups is essential for ensuring fairness and reliability in real-world applications …

A central goal of machine learning is to learn robust representations that capture the
fundamental relationship between inputs and output labels. However, minimizing training …

Understanding the performance of machine learning models across diverse data
distributions is critically important for reliable applications. Motivated by this, there is a …

Classifiers in machine learning are often brittle when deployed. Particularly concerning are
models with inconsistent performance on specific subgroups of a class, eg, exhibiting …

Prior work has shown that Visual Recognition datasets frequently underrepresent bias
groups B (eg Female) within class labels Y (eg Programmers). This dataset bias can lead to …

Overparameterized neural networks can be highly accurate on average on an iid test set yet
consistently fail on atypical groups of the data (eg, by learning spurious correlations that …