This work describes a fully automatic technique to calibrate a geometric mapping between lidar and video feeds on a mobile ground-based platform. This data association is a crucial first step for any multi-modal scene understanding system which aims to leverage the complementary information of the two sensors. While several systems have been previously described which use hand-calibration or specific scenery to achieve this goal, the system described here is fully automatic and generates an accurate association without user intervention or calibration objects. The estimated parameters include the 7 classical camera parameters for a linear pinhole model, i.e., rotation, position, and focal length parameters, as well as an estimation of the radial distortion. The system uses a multi stage process to bootstrap difficult parameters based on robust estimates of easier ones. The calibration algorithm is tested empirically using free online data supplied as part of the DARPA Urban Challenge autonomous vehicle competition. Experiments are performed to illustrate the stability, and computation cost of the algorithm.