Understanding the performance of machine learning models across diverse data
distributions is critically important for reliable applications. Motivated by this, there is a …

Dataset shift is a frequent cause of failure of a predictor. A model which performs well in
several contexts can give bad predictions in other contexts where the data are shifted …

We develop a methodology for assessing the robustness of models to subpopulation shift---
specifically, their ability to generalize to novel data subpopulations that were not observed …

Distribution shift is a major source of failure for machine learning models. However,
evaluating model reliability under distribution shift can be challenging, especially since it …

A central goal of machine learning is to learn robust representations that capture the
fundamental relationship between inputs and output labels. However, minimizing training …

We introduce four new real-world distribution shift datasets consisting of changes in image
style, image blurriness, geographic location, camera operation, and more. With our new …

Machine learning systems deployed in the wild are often trained on a source distribution but
deployed on a different target distribution. Unlabeled data can be a powerful point of …

Robustness to distribution shifts is critical for deploying machine learning models in the real
world. Despite this necessity, there has been little work in defining the underlying …

Existing methods for isolating hard subpopulations and spurious correlations in datasets
often require human intervention. This can make these methods labor-intensive and dataset …

Distribution shifts—where the training distribution differs from the test distribution—can
substantially degrade the accuracy of machine learning (ML) systems deployed in the wild …