On the road with gpt-4v (ision): Early explorations of visual-language model on autonomous driving

L Wen, X Yang, D Fu, X Wang, P Cai, X Li, T Ma… - arXiv preprint arXiv …, 2023 - arxiv.org
The pursuit of autonomous driving technology hinges on the sophisticated integration of
perception, decision-making, and control systems. Traditional approaches, both data-driven …

Drivevlm: The convergence of autonomous driving and large vision-language models

X Tian, J Gu, B Li, Y Liu, C Hu, Y Wang, K Zhan… - arXiv preprint arXiv …, 2024 - arxiv.org
A primary hurdle of autonomous driving in urban environments is understanding complex
and long-tail scenarios, such as challenging road conditions and delicate human behaviors …

Drivegpt4: Interpretable end-to-end autonomous driving via large language model

Z Xu, Y Zhang, E Xie, Z Zhao, Y Guo, KKY Wong… - arXiv preprint arXiv …, 2023 - arxiv.org
In the past decade, autonomous driving has experienced rapid development in both
academia and industry. However, its limited interpretability remains a significant unsolved …

Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving

V Dewangan, T Choudhary, S Chandhok… - arXiv preprint arXiv …, 2023 - arxiv.org
Talk2BEV is a large vision-language model (LVLM) interface for bird's-eye view (BEV) maps
in autonomous driving contexts. While existing perception systems for autonomous driving …

Vision language models in autonomous driving and intelligent transportation systems

X Zhou, M Liu, BL Zagar, E Yurtsever… - arXiv preprint arXiv …, 2023 - arxiv.org
The applications of Vision-Language Models (VLMs) in the fields of Autonomous Driving
(AD) and Intelligent Transportation Systems (ITS) have attracted widespread attention due to …

St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning

S Hu, L Chen, P Wu, H Li, J Yan, D Tao - European Conference on …, 2022 - Springer
Many existing autonomous driving paradigms involve a multi-stage discrete pipeline of
tasks. To better predict the control signals and enhance user safety, an end-to-end approach …

Dolphins: Multimodal language model for driving

Y Ma, Y Cao, J Sun, M Pavone, C Xiao - arXiv preprint arXiv:2312.00438, 2023 - arxiv.org
The quest for fully autonomous vehicles (AVs) capable of navigating complex real-world
scenarios with human-like understanding and responsiveness. In this paper, we introduce …

Lmdrive: Closed-loop end-to-end driving with large language models

H Shao, Y Hu, L Wang, G Song… - Proceedings of the …, 2024 - openaccess.thecvf.com
Despite significant recent progress in the field of autonomous driving modern methods still
struggle and can incur serious accidents when encountering long-tail unforeseen events …

Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models

X Ding, J Han, H Xu, X Liang… - Proceedings of the …, 2024 - openaccess.thecvf.com
The rise of multimodal large language models (MLLMs) has spurred interest in language-
based driving tasks. However existing research typically focuses on limited tasks and often …

[PDF][PDF] Drive like a human: Rethinking autonomous driving with large language models

D Fu, X Li, L Wen, M Dou, P Cai… - Proceedings of the …, 2024 - openaccess.thecvf.com
In this paper, we explore the potential of using a large language model (LLM) to understand
the driving environment in a human-like manner and analyze its ability to reason, interpret …