In the study of numerical and arithmetical abilities, there is compelling evidence demonstrating that number and space representations are connected to one another. Historically the first source of support came more than a century ago, when Galton's investigations on mental imagery suggested that the internal representation of numbers may evoke a stable, linear space. In the past few decades, empirical evidence lent further support to the hypothesis that numerical representation is spatially coded into a non-verbal ‘mental number line’, which in turn lead to considering this representation as the core of number meaning. Visuo-spatial processing is intuitively involved in various aspects of number processing and calculation: For instance, the meaning of a digit in a multi-digit number is coded following spatial information, given its association to its relative position within the number; similarly, to solve a complex written multiplication one has to know the correct location of the intermediate results. In this review behavioral, neuropsychological, and neuroimaging data concerning the close relationship between numerical abilities and visuo-spatial processes are considered.