The bias-variance decomposition is a very useful and widely-used tool for understanding
machine-learning algorithms. It was originally developed for squared loss. In recent years …

Machine learning has come of age. And just in case you might think this is a mere platitude,
let me clarify. The dream that machines would one day be able to learn is as old as …

We study the problem of (learning) algorithm comparison, where the goal is to find
differences between models trained with two different learning algorithms. We begin by …

Classifier learning methods commonly assume that the training data consist of randomly
drawn examples from the same distribution as the test examples about which the learned …

Training a learning machine from examples is accomplished by minimizing a quantitative
error measure, the training error defined over a training set. A low error on the training set …

The machine learning community has become increasingly concerned with the potential for
bias and discrimination in predictive models. This has motivated a growing line of work on …

ROC analysis investigates and employs the relationship between sensitivity and specificity
of a binary classifier. Sensitivity or true positiverate measures the proportion of positives …

This paper introduces cost curves, a graphical technique for visualizing the performance
(error rate or expected cost) of 2-class classifiers over the full range of possible class …

Training classifiers on large databases is computationally demand (cid: 173) ing. It is
desirable to develop efficient procedures for a reliable prediction of a classifier's suitability …

The area under the ROC curve (AUC), a wellknown measure of ranking performance, is also
often used as a measure of classification performance, aggregating over decision thresholds …