Using unmanned aerial vehicles (UAV) as devices for traffic data collection exhibits many advantages in collecting traffic information. This paper introduces a new vehicle detecting and tracking system based on image data collected by UAV. This system uses consecutive frames to generate vehicle's dynamic information, such as positions and velocities. Four major modules have been developed: image registration, image feature extraction, vehicle shape detecting, and vehicle tracking. Some unique features have been introduced into this system to customize the vehicle and traffic flow and to jointly use them in multiple consecutive images to increase the system accuracy of detecting and tracking vehicles. Field tests demonstrate that the present system exhibits high accuracy in traffic information acquisition at different UAV altitudes with different view scopes, which can be used in future traffic monitoring and control in metropolitan areas.