Attentional set shifting is a core ingredient of cognition, allowing for fast adaptation to changes in the environment. How this skill compares between humans and other primates is not well known. We examined performance of 3- to 5-year-old children and chimpanzees on a new attentional set shifting task. We presented participants with two shelves holding the same set of four boxes. To choose the correct box on each shelf, one has to switch attention depending on which shelf one is currently presented with. Experiment 1 (forty-six 3- to 5-year olds, predominantly European White) established content validity, showing that the majority of errors were specific switching mistakes indicating failure to shift attention. Experiment 2 (one hundred and seventy-eight 3- to 6-year olds, predominantly European White) showed that older children made fewer mistakes, but if mistakes were made, a larger proportion were switching mistakes rather than ‘random’ errors. Experiment 3 (52 chimpanzees) established suitability of the task for non-human great apes and showed that chimpanzees' performance was comparable to the performance of 3- and 4-year olds, but worse than 5-year olds. These results suggest that chimpanzees and young children share attentional set shifting capacities, but that there are unique changes in the human lineage from 5 years of age.