Low-level visual perception deteriorates during healthy aging. We hypothesized that age-related retinal and cortical structure deteriorations affect perception through specific disruptions of neural function. We measured perceptual visual acuity in fifty healthy adults aged 20-80 years. We then measured these participants’ early visual field map (V1, V2 and V3) functional population receptive field (pRF) sizes and structural surface areas using fMRI, and their retinal structure using high-definition optical coherence tomography. With increasing age visual acuity decreased, pRF sizes increased, visual field maps surface areas decreased, and retinal thickness decreased. Among these measures, only functional pRF sizes predicted perceptual visual acuity. PRF sizes were in turn predicted by cortical structure only (surface areas), which were only predicted by retinal structure (thickness). We propose that age-related retinal structural deterioration disrupts cortical structure, thereby disrupting cortical functional neural interactions that normally sharpen visual position selectivity: the resulting functional disruption underlies age-related perceptual deterioration.