Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops. In this …
We present a conceptual framework, datamodeling, for analyzing the behavior of a model class in terms of the training data. For any fixed" target" example $ x $, training set $ S $, and …
T LaBonte, V Muthukumar… - Advances in Neural …, 2024 - proceedings.neurips.cc
Empirical risk minimization (ERM) of neural networks is prone to over-reliance on spurious correlations and poor generalization on minority groups. The recent deep feature …
J Chen, F Liu, B Avci, X Wu… - Advances in Neural …, 2021 - proceedings.neurips.cc
When a deep learning model is deployed in the wild, it can encounter test data drawn from distributions different from the training data distribution and suffer drop in performance. For …
H Shah, SM Park, A Ilyas… - … Conference on Machine …, 2023 - proceedings.mlr.press
We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms. We begin by …
Despite the emergence of principled methods for domain adaptation under label shift, their sensitivity to shifts in class conditional distributions is precariously under explored …
Evaluating the performance of graph neural networks (GNNs) is an essential task for practical GNN model deployment and serving, as deployed GNNs face significant …
H Yu, J Liu, X Zhang, J Wu, P Cui - arXiv preprint arXiv:2403.01874, 2024 - arxiv.org
Machine learning models, while progressively advanced, rely heavily on the IID assumption, which is often unfulfilled in practice due to inevitable distribution shifts. This renders them …
Y Yu, Z Yang, A Wei, Y Ma… - … Conference on Machine …, 2022 - proceedings.mlr.press
We propose a metric—Projection Norm—to predict a model's performance on out-of- distribution (OOD) data without access to ground truth labels. Projection Norm first uses …