Different evaluation measures assess different characteristics of machine learning
algorithms. The empirical evaluation of algorithms and classifiers is a matter of on-going …

The evaluation of classifiers or learning algorithms is not a topic that has, generally, been
given much thought in the fields of Machine Learning and Data Mining. More often than not …

This paper gives an overview of some ways in which our understanding of performance
evaluation measures for machine-learned classifiers has improved over the last twenty …

In this note we use examples from the literature to illustrate some poor practices in
assessing the performance of supervised classification rules, and we suggest guidelines for …

The area under the ROC curve (AUC) is a very widely used measure of performance for
classification and diagnostic rules. It has the appealing property of being objective, requiring …

The selection of the best classification algorithm for a given dataset is a very widespread
problem. It is also a complex one, in the sense it requires to make several important …

This paper introduces cost curves, a graphical technique for visualizing the performance
(error rate or expected cost) of 2-class classifiers over the full range of possible class …

The field of machine learning has matured to the point where many sophisticated learning
approaches can be applied to practical applications. Thus it is of critical importance that …

Performance metrics in classification are fundamental in assessing the quality of learning
methods and learned models. However, many different measures have been defined in the …

We analyze critically the use of classification accuracy to compare classifiers on natural data
sets, providing a thorough investigation using ROC analysis, standard machine learning …