Recently, Miller et al. showed that a model's in-distribution (ID) accuracy has a strong linear
correlation with its out-of-distribution (OOD) accuracy, on several OOD benchmarks, a …

Recent work has shown that the accuracy of machine learning models can vary substantially
when evaluated on a distribution that even slightly differs from that of the training data. As a …

Real-world machine learning deployments are characterized by mismatches between the
source (training) and target (test) distributions that may cause performance drops. In this …

Large-scale vision-language models (VLMs) like CLIP successfully find correspondences
between images and text. Through the standard deterministic mapping process, an image or …

When a deep learning model is deployed in the wild, it can encounter test data drawn from
distributions different from the training data distribution and suffer drop in performance. For …

Evaluating the performance of graph neural networks (GNNs) is an essential task for
practical GNN model deployment and serving, as deployed GNNs face significant …

Machine learning models, while progressively advanced, rely heavily on the IID assumption,
which is often unfulfilled in practice due to inevitable distribution shifts. This renders them …

We propose a metric—Projection Norm—to predict a model's performance on out-of-
distribution (OOD) data without access to ground truth labels. Projection Norm first uses …

Abstract The Automated Model Evaluation (AutoEval) framework entertains the possibility of
evaluating a trained machine learning model without resorting to a labeled testing set …

Abstract Out-of-distribution (OOD) data poses serious challenges in deployed machine
learning models, so methods of predicting a model's performance on OOD data without …