JAWS-x: Addressing efficiency bottlenecks of conformal prediction under standard and feedback covariate shift

D Prinster, S Saria, A Liu - International Conference on …, 2023 - proceedings.mlr.press
We study the efficient estimation of predictive confidence intervals for black-box predictors
when the common data exchangeability (eg, iid) assumption is violated due to potentially …

Density-Regression: Efficient and Distance-Aware Deep Regressor for Uncertainty Estimation under Distribution Shifts

HM Bui, A Liu - International Conference on Artificial …, 2024 - proceedings.mlr.press
Morden deep ensembles technique achieves strong uncertainty estimation performance by
going through multiple forward passes with different models. This is at the price of a high …

A data-driven framework for identifying patient subgroups on which an AI/machine learning model may underperform

A Subbaswamy, B Sahiner, N Petrick, V Pai… - npj Digital …, 2024 - nature.com
A fundamental goal of evaluating the performance of a clinical model is to ensure it performs
well across a diverse intended patient population. A primary challenge is that the data used …

Trustworthy Transfer Learning: A Survey

J Wu, J He - arXiv preprint arXiv:2412.14116, 2024 - arxiv.org
Transfer learning aims to transfer knowledge or information from a source domain to a
relevant target domain. In this paper, we understand transfer learning from the perspectives …

Training-Conditional Coverage Bounds under Covariate Shift

M Pournaderi, Y Xiang - arXiv preprint arXiv:2405.16594, 2024 - arxiv.org
Training-conditional coverage guarantees in conformal prediction concern the concentration
of the error distribution, conditional on the training data, below some nominal level. The …

Distribution-free risk assessment of regression-based machine learning algorithms

S Singh, N Sarna, Y Li, Y Li, A Orfanoudaki… - arXiv preprint arXiv …, 2023 - arxiv.org
Machine learning algorithms have grown in sophistication over the years and are
increasingly deployed for real-life applications. However, when using machine learning …

[PDF][PDF] Efficient Approximate Predictive Inference Under Feedback Covariate Shift with Influence Functions

D Prinster, S Saria, A Liu - Conformal and Probabilistic …, 2023 - proceedings.mlr.press
We propose JAWA-FCS, which uses higher-order influence functions to approximate
predictive intervals of the (previous) jackknife+ weighted for feedback covariate shift for …

Robust Conformal Prediction Using Privileged Information

S Feldman, Y Romano - arXiv preprint arXiv:2406.05405, 2024 - arxiv.org
We develop a method to generate prediction sets with a guaranteed coverage rate that is
robust to corruptions in the training data, such as missing or noisy variables. Our approach …

Distribution-Free Predictive Inference under Unknown Temporal Drift

E Han, C Huang, K Wang - arXiv preprint arXiv:2406.06516, 2024 - arxiv.org
Distribution-free prediction sets play a pivotal role in uncertainty quantification for complex
statistical models. Their validity hinges on reliable calibration data, which may not be readily …

Conformal Validity Guarantees Exist for Any Data Distribution

D Prinster, S Stanton, A Liu, S Saria - arXiv preprint arXiv:2405.06627, 2024 - arxiv.org
As machine learning (ML) gains widespread adoption, practitioners are increasingly seeking
means to quantify and control the risk these systems incur. This challenge is especially …