Abstract Model selection strategies for machine learning algorithms typically involve the
numerical optimisation of an appropriate model selection criterion, often based on an …

The correct use of model evaluation, model selection, and algorithm selection techniques is
vital in academic machine learning research as well as in many industrial settings. This …

Breakthroughs in machine learning are rapidly changing science and society, yet our
fundamental understanding of this technology has lagged far behind. Indeed, one of the …

We analyze the performance of cross validation 1 in the context of model selection and
complexity regularization. We work in a setting in which we must choose the right number of …

Most machine learning researchers perform quantitative experiments to estimate
generalization error and compare algorithm performances. In order to draw statistically …

Selecting a good model of a set of input points by cross validation is a computationally
intensive process, especially if the number of possible models or the number of training …

This paper gives an overview of some ways in which our understanding of performance
evaluation measures for machine-learned classifiers has improved over the last twenty …

When performing a regression or classification analysis, one needs to specify a statistical
model. This model should avoid the overfitting and underfitting of data, and achieve a low …

Many performance metrics have been introduced in the literature for the evaluation of
classification performance, each of them with different origins and areas of application …

Abstract Model selection is an important ingredient of many machine learning algorithms, in
particular when the sample size in small, in order to strike the right trade-off between …