This paper describes a real-time road following and road junction detection vision system for autonomous vehicles. Vision-guided road following requires extracting road boundaries from images in real-time to guide the navigation of autonomous vehicles on the roadway. We use a histogram-based pixel classification algorithm to classify road and non-road regions in the image. The most likely road region is selected and a polygonal representation of the detected road region boundary is used as the input to a geometric reasoning module that performs model-based reasoning to accurately identify consistent road segments and road junctions. In this module, local geometric supports for each road edge segment are collected and recorded and a global consistency checking is performed' to obtain a consistent interpretation of the raw data. Limited cases of incorrect image segmentation due to shadows or unusual road conditions can be detected and corrected based on the road model. Similarly, road junctions can be detected using the same principle. The real-time road following vision system has been implemented on a high-speed image processor connected to a host computer. We have tested our road following vision system and vehicle control system on a gravel road.