Calibrating deep neural networks using focal loss

J Mukhoti, V Kulharia, A Sanyal… - Advances in …, 2020 - proceedings.neurips.cc
Miscalibration--a mismatch between a model's confidence and its correctness--of Deep
Neural Networks (DNNs) makes their predictions hard to rely on. Ideally, we want networks …

Improving model calibration with accuracy versus uncertainty optimization

R Krishnan, O Tickoo - Advances in Neural Information …, 2020 - proceedings.neurips.cc
Obtaining reliable and accurate quantification of uncertainty estimates from deep neural
networks is important in safety-critical applications. A well-calibrated model should be …

Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning

J Zhang, B Kailkhura, TYJ Han - International conference on …, 2020 - proceedings.mlr.press
This paper studies the problem of post-hoc calibration of machine learning classifiers. We
introduce the following desiderata for uncertainty calibration:(a) accuracy-preserving,(b) …

Calibration of neural networks using splines

K Gupta, A Rahimi, T Ajanthan, T Mensink… - arXiv preprint arXiv …, 2020 - arxiv.org
Calibrating neural networks is of utmost importance when employing them in safety-critical
applications where the downstream decision making depends on the predicted probabilities …

Calibrated language model fine-tuning for in-and out-of-distribution data

L Kong, H Jiang, Y Zhuang, J Lyu, T Zhao… - arXiv preprint arXiv …, 2020 - arxiv.org
Fine-tuned pre-trained language models can suffer from severe miscalibration for both in-
distribution and out-of-distribution (OOD) data due to over-parameterization. To mitigate this …

Non-parametric calibration for classification

J Wenger, H Kjellström… - … Conference on Artificial …, 2020 - proceedings.mlr.press
Many applications of classification methods not only require high accuracy but also reliable
estimation of predictive uncertainty. However, while many current classification frameworks …

Improved trainable calibration method for neural networks on medical imaging classification

G Liang, Y Zhang, X Wang, N Jacobs - arXiv preprint arXiv:2009.04057, 2020 - arxiv.org
Recent works have shown that deep neural networks can achieve super-human
performance in a wide range of image classification tasks in the medical imaging domain …

Distribution-free binary classification: prediction sets, confidence intervals and calibration

C Gupta, A Podkopaev… - Advances in Neural …, 2020 - proceedings.neurips.cc
We study three notions of uncertainty quantification---calibration, confidence intervals and
prediction sets---for binary classification in the distribution-free setting, that is without making …

Intra order-preserving functions for calibration of multi-class neural networks

A Rahimi, A Shaban, CA Cheng… - Advances in Neural …, 2020 - proceedings.neurips.cc
Predicting calibrated confidence scores for multi-class deep networks is important for
avoiding rare but costly mistakes. A common approach is to learn a post-hoc calibration …

Hardware design and the competency awareness of a neural network

Y Ding, W Jiang, Q Lou, J Liu, J Xiong, XS Hu, X Xu… - Nature …, 2020 - nature.com
The ability to estimate the uncertainty of predictions made by a neural network is essential
when applying neural networks to tasks such as medical diagnosis and autonomous …