We present a model for the autonomous and simultaneous learning of smooth pursuit and vergence eye movements based on principles of efficient coding. The model accounts for the joint development of visual encoding and eye movement control. Sparse coding models encode the incoming data and capture the statistics of the input in spatio-temporal basis functions while a reinforcement learner generates eye movements to optimise the efficiency of the encoding. We consider the embodiment of the approach in the iCub simulator and demonstrate the emergence of a self-calibrating smooth pursuit and vergence behaviour. Unlike standard computer vision approaches, it is driven by the interaction between sensory encoding and eye movements. Interestingly, our analysis shows that the emerging representations learned by this model are in line with results on velocity and disparity tuning properties of neurons in visual cortex.