subjective assessment tests. Apparent differences among subjects can, for the most part, be
explained by random factors. Building on that theory, we propose a subjective test design
where four to six team members each rate the stimuli multiple times. The results are
comparable to a high performing objective metric. This provides a quick and simple way to
analyze new technologies and perform pre-tests for subjective assessment.