Diabetic macular edema is one of the retinal abnormalities which affects the central vision of the person and causes total blindness in severe cases. Fovea (center of macula) localization is an important step in retinal image analysis especially for grading diabetic macular edema. This paper describes a method to automatically localize the fovea center in retinal fundus images. The method is mainly based on mathematical morphology along with the information of other anatomic structures such as blood vessel and optic disc. Initially, the vascular structure and optic disc center are extracted, and then the morphological operations are employed on the gray scale image of green channel for fovea candidates’ selection. The candidates’ satisfying area, density and distance criteria are considered for the final stage. In the final stage, the candidate having lesser vessel pixels was considered as fovea region. The proposed method was evaluated on the two publicly available DRIVE and STARE databases. The method was able to obtain 100% of fovea localization accuracy on DRIVE database with 2.88 seconds average computation time.