Two-stage learning to defer with multiple experts

A Mao, C Mohri, M Mohri… - Advances in neural …, 2024 - proceedings.neurips.cc
We study a two-stage scenario for learning to defer with multiple experts, which is crucial in
practice for many applications. In this scenario, a predictor is derived in a first stage by …

Theoretically grounded loss functions and algorithms for score-based multi-class abstention

A Mao, M Mohri, Y Zhong - International Conference on …, 2024 - proceedings.mlr.press
Learning with abstention is a key scenario where the learner can abstain from making a
prediction at some cost. In this paper, we analyze the score-based formulation of learning …

Who should predict? exact algorithms for learning to defer to humans

H Mozannar, H Lang, D Wei… - International …, 2023 - proceedings.mlr.press
Automated AI classifiers should be able to defer the prediction to a human decision maker to
ensure more accurate predictions. In this work, we jointly train a classifier with a rejector …

Predictor-rejector multi-class abstention: Theoretical analysis and algorithms

A Mao, M Mohri, Y Zhong - International Conference on …, 2024 - proceedings.mlr.press
We study the key framework of learning with abstention in the multi-class classification
setting. In this setting, the learner can choose to abstain from making a prediction with some …

In defense of softmax parametrization for calibrated and consistent learning to defer

Y Cao, H Mozannar, L Feng… - Advances in Neural …, 2024 - proceedings.neurips.cc
Enabling machine learning classifiers to defer their decision to a downstream expert when
the expert is more accurate will ensure improved safety and performance. This objective can …

Generalizing consistent multi-class classification with rejection to be compatible with arbitrary losses

Y Cao, T Cai, L Feng, L Gu, J Gu, B An… - Advances in neural …, 2022 - proceedings.neurips.cc
Abstract\emph {Classification with rejection}(CwR) refrains from making a prediction to avoid
critical misclassification when encountering test samples that are difficult to classify. Though …

Post-hoc estimators for learning to defer to an expert

H Narasimhan, W Jitkrittum… - Advances in …, 2022 - proceedings.neurips.cc
Many practical settings allow a learner to defer predictions to one or more costly experts. For
example, the learning to defer paradigm allows a learner to defer to a human expert, at …

Principled approaches for learning to defer with multiple experts

A Mao, M Mohri, Y Zhong - International Workshop on Combinatorial …, 2024 - Springer
We present a study of surrogate losses and algorithms for the general problem of learning to
defer with multiple experts. We first introduce a new family of surrogate losses specifically …

Learning to defer to multiple experts: Consistent surrogate losses, confidence calibration, and conformal ensembles

R Verma, D Barrejón… - … Conference on Artificial …, 2023 - proceedings.mlr.press
We study the statistical properties of learning to defer (L2D) to multiple experts. In particular,
we address the open problems of deriving a consistent surrogate loss, confidence …

Unified classification and rejection: A one-versus-all framework

Z Cheng, XY Zhang, CL Liu - Machine Intelligence Research, 2024 - Springer
Classifying patterns of known classes and rejecting ambiguous and novel (also called as out-
of-distribution (OOD)) inputs are involved in open world pattern recognition. Deep neural …