Many real-world data-mining applications involve obtaining predictive models using
datasets with strongly imbalanced distributions of the target variable. Frequently, the least …

Abstract Machine learning (ML) offers robust statistical and probabilistic techniques that can
help to make sense of large amounts of data. This scoping review paper aims to broadly …

Accurate estimation of predictive uncertainty (model calibration) is essential for the safe
application of neural networks. Many instances of miscalibration in modern neural networks …

Even if measuring the outcome of binary classifications is a pivotal task in machine learning
and statistics, no consensus has been reached yet about which statistical rate to employ to …

In pursuit of the perfect supervised NLP classifier, razor thin margins and low-resource test
sets can make modeling decisions difficult. Popular metrics such as Accuracy, Precision …

Our goal was to apply a statistical approach to allow the identification of atypical language
patterns and to differentiate patients with epilepsy from healthy subjects, based on their …

Current benchmark reports of classification algorithms generally concern common classifiers
and their variants but do not include many algorithms that have been introduced in recent …

Precision-Recall analysis abounds in applications of binary classification where true
negatives do not add value and hence should not affect assessment of the classifier's …

Background. The ability to predict defect-prone software components would be valuable.
Consequently, there have been many empirical studies to evaluate the performance of …

This paper gives an overview of some ways in which our understanding of performance
evaluation measures for machine-learned classifiers has improved over the last twenty …