This paper introduces a novel strategy for the task of simultaneously locating two key anatomical landmarks in retinal images of the eye fundus, namely the optic disc and the fovea. For that, instead of attempting to classify each pixel as belonging to the background, the optic disc, or the fovea center, which would lead to a highly class-imbalanced setting, the problem is reformulated as a pixelwise regression task. The regressed quantity consists of the distance from the closest landmark of interest. A Fully-Convolutional Deep Neural Network is optimized to predict this distance for each image location, implicitly casting the problem into a per-pixel Multi-Task Learning approach by which a globally consistent distribution of distances across the entire image can be learned. Once trained, the two minimal distances predicted by the model are selected as the locations of the optic disc and the fovea. The joint learning of every pixel position relative to the optic disc and the fovea favors an automatic understanding of the overall anatomical distribution. This results in an effective technique that can detect both locations simultaneously, as opposed to previous methods that handle both tasks separately. Comprehensive experimental results on a large public dataset validate the proposed approach.