The evaluation of artificial intelligence systems and components is crucial for the progress of
the discipline. In this paper we describe and critically assess the different ways AI systems …

While some computational models of intelligence test problems were proposed throughout
the second half of the XXth century, in the first years of the XXIst century we have seen an …

Complex, but specific, tasks—such as chess or Jeopardy!—are popularly seen as
milestones for artificial intelligence (AI). However, they are not appropriate for evaluating the …

Comparing humans and machines is one important source of information about both
machine and human strengths and limitations. Most of these comparisons and competitions …

We present and develop the notion of 'universal psychometrics' as a subject of study, and
eventually a discipline, that focusses on the measurement of cognitive abilities for the …

In the last 20 years the Turing test has been left further behind by new developments in
artificial intelligence. At the same time, however, these developments have revived some …

The notion of a universal intelligence test has been recently advocated as a means to
assess humans, non-human animals and machines in an integrated, uniform way. While the …

Inspired by recent progress in multi-agent Reinforcement Learning (RL), in this work we
examine the collective intelligent behaviour of theoretical universal agents by introducing a …

Evaluating agent intelligence is a fundamental issue for the understanding, construction and
improvement of autonomous agents. New intelligence tests have been recently developed …

Animals, including humans, are usually judged on what they could become, rather than what
they are. Many physical and cognitive abilities in the 'animal kingdom'are only acquired (to a …