Security of (semi)-autonomous vehicles is a growing concern, first, due to the increased exposure of the functionality to potential attackers; second, due to the reliance of functionalities on diverse (semi)-autonomous systems; third, due to the interaction of a single-vehicle with myriads of other smart systems in urban traffic infrastructure. Beyond these technical issues, we argue that the security-by-design principle for smart and complex autonomous systems, such as an Autonomous Vehicle (AV) is poorly understood and rarely practiced. Unlike traditional IT systems, where the risk mitigation techniques and adversarial models are well studied and developed with security design principles such as security perimeter and defense-in-depth, the lack of such a framework for connected autonomous systems is plaguing the design and implementation of a secure AV. We attempt to identify the core issues of securing an AV. This is done methodically by developing a security-by-design framework for AV from the first principles. Subsequently, the technical challenges for AV security are identified.