The past decade has witnessed significant breakthroughs in autonomous driving technologies. We are heading toward an intelligent and efficient transportation system where human errors are eliminated. While excited about the emergence of autonomous vehicles with increasing intelligence, the public has also raised concerns about their reliability. Modern autonomous driving systems usually adopt black-box deep-learning models for multiple function modules (eg, perception, behavior prediction, behavior generation). The opaque nature of neural networks and their complex system architecture make it extremely difficult to understand the behavior of the overall system, which prevents humans from confidingly sharing the road and interacting with autonomous vehicles. This motivates the design of a more transparent system to build a foundation for trustworthy interaction between humans and autonomous vehicles.