In a recently published paper in JMLR, Demšar (2006) recommends a set of non-parametric
statistical tests and procedures which can be safely used for comparing the performance of …

While methods for comparing two learning algorithms on a single data set have been
scrutinized for quite some time already, the issue of statistical tests for comparisons of more …

A popular procedure for identifying which one of two classification algorithms has a better
performance is to test them on multiple data sets, and the accuracies resulting from k-fold …

To prove that a certain algorithm is superior to the benchmark algorithms, the statistical
hypothesis tests are commonly adopted with experimental results on a number of datasets …

Experimental analysis of the performance of a proposed method is a crucial and necessary
task in an investigation. In this paper, we focus on the use of nonparametric statistical …

Dietterich (1998) reviews five statistical tests and proposes the 5× 2 cv t test for determining
whether there is a significant difference between the error rates of two classifiers. In our …

This paper presents a method for combining classifiers that uses estimates of each
individual classifier's local accuracy in small regions of feature space surrounding an …

Classification accuracy as a proxy for two-sample testing Page 1 The Annals of Statistics
2021, Vol. 49, No. 1, 411–434 https://doi.org/10.1214/20-AOS1962 © Institute of Mathematical …

Binary classification is a well studied special case of the classification problem. Statistical
properties of binary classifiers, such as consistency, have been investigated in a variety of …

We explore the framework of permutation-based p-values for assessing the performance of
classifiers. In this paper we study two simple permutation tests. The first test assess whether …