This paper presents a simulation of vehicle driving control system in terms of lateral and longitudinal control using image processing. The main contribution of this study is it contributes an algorithm of vehicle lane detection and tracking which based on colour cue segmentation, Canny edge detection and Hough transform. The algorithm gave good result in detecting straight and smooth curvature lane on highway even the lane was affected by shadow. Then by combining and processing the result of lane detection process with vehicle dynamics model, this system will produce the dynamics performance of vehicle driving control. This simulation system was divided into four subsystems: sensor, image processing, controller and vehicle. All the methods have been tested on video data and the experimental results have demonstrated a fast and robust system.