Since the potential of soft computing for driver assistance systems has been recognized, much effort has been spent in the development of appropriate techniques for robust lane detection, object classification, tracking, and representation of task relevant objects. For such systems in order to be able to perform their tasks the environment must be sensed by one or more sensors. Usually a complex processing, fusion, and interpretation of the sensor data is required and imposes a modular architecture for the overall system. In this paper, we present specific approaches considering the main components of such systems. We concentrate on image processing as the main source of relevant object information, representation and fusion of data that might arise from different sensors, and behavior planning and generation as a basis for autonomous driving. Within our system components most paradigms of soft computing are employed; in this article we focus on Kalman filtering for sensor fusion, neural field dynamics for behavior generation, and evolutionary algorithms for optimization of parts of the system.