We develop a methodology for assessing the robustness of models to subpopulation shift---
specifically, their ability to generalize to novel data subpopulations that were not observed …

Machine learning models often perform poorly on subgroups that are underrepresented in
the training data. Yet, little is understood on the variation in mechanisms that cause …

Understanding the performance of machine learning models across diverse data
distributions is critically important for reliable applications. Motivated by this, there is a …

Machine learning systems deployed in the wild are often trained on a source distribution but
deployed on a different target distribution. Unlabeled data can be a powerful point of …

Robustness to distribution shifts is critical for deploying machine learning models in the real
world. Despite this necessity, there has been little work in defining the underlying …

When transferring a pretrained model to a downstream task, two popular methods are full
fine-tuning (updating all the model parameters) and linear probing (updating only the last …

Distribution shifts—where the training distribution differs from the test distribution—can
substantially degrade the accuracy of machine learning (ML) systems deployed in the wild …

Prior work has shown that Visual Recognition datasets frequently underrepresent bias
groups B (eg Female) within class labels Y (eg Programmers). This dataset bias can lead to …

Existing methods for isolating hard subpopulations and spurious correlations in datasets
often require human intervention. This can make these methods labor-intensive and dataset …

Subpopulation shift widely exists in many real-world machine learning applications, referring
to the training and test distributions containing the same subpopulation groups but varying in …