Recently, Miller et al. showed that a model's in-distribution (ID) accuracy has a strong linear
correlation with its out-of-distribution (OOD) accuracy, on several OOD benchmarks, a …

Real-world machine learning deployments are characterized by mismatches between the
source (training) and target (test) distributions that may cause performance drops. In this …

We present a conceptual framework, datamodeling, for analyzing the behavior of a model
class in terms of the training data. For any fixed" target" example $ x $, training set $ S $, and …

Empirical risk minimization (ERM) of neural networks is prone to over-reliance on spurious
correlations and poor generalization on minority groups. The recent deep feature …

When a deep learning model is deployed in the wild, it can encounter test data drawn from
distributions different from the training data distribution and suffer drop in performance. For …

We study the problem of (learning) algorithm comparison, where the goal is to find
differences between models trained with two different learning algorithms. We begin by …

Despite the emergence of principled methods for domain adaptation under label shift, their
sensitivity to shifts in class conditional distributions is precariously under explored …

Evaluating the performance of graph neural networks (GNNs) is an essential task for
practical GNN model deployment and serving, as deployed GNNs face significant …

Machine learning models, while progressively advanced, rely heavily on the IID assumption,
which is often unfulfilled in practice due to inevitable distribution shifts. This renders them …

We propose a metric—Projection Norm—to predict a model's performance on out-of-
distribution (OOD) data without access to ground truth labels. Projection Norm first uses …