Gpt-driver: Learning to drive with gpt

J Mao, Y Qian, H Zhao, Y Wang - arXiv preprint arXiv:2310.01415, 2023 - arxiv.org
We present a simple yet effective approach that can transform the OpenAI GPT-3.5 model
into a reliable motion planner for autonomous vehicles. Motion planning is a core challenge …

A survey on multimodal large language models for autonomous driving

C Cui, Y Ma, X Cao, W Ye, Y Zhou… - Proceedings of the …, 2024 - openaccess.thecvf.com
With the emergence of Large Language Models (LLMs) and Vision Foundation Models
(VFMs), multimodal AI systems benefiting from large models have the potential to equally …

Lampilot: An open benchmark dataset for autonomous driving with language model programs

Y Ma, C Cui, X Cao, W Ye, P Liu, J Lu… - Proceedings of the …, 2024 - openaccess.thecvf.com
Autonomous driving (AD) has made significant strides in recent years. However existing
frameworks struggle to interpret and execute spontaneous user instructions such as" …

LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning

S Chen, X Chen, C Zhang, M Li, G Yu… - Proceedings of the …, 2024 - openaccess.thecvf.com
Abstract Recent progress in Large Multimodal Models (LMM) has opened up great
possibilities for various applications in the field of human-machine interactions. However …

Dilu: A knowledge-driven approach to autonomous driving with large language models

L Wen, D Fu, X Li, X Cai, T Ma, P Cai, M Dou… - arXiv preprint arXiv …, 2023 - arxiv.org
Recent advancements in autonomous driving have relied on data-driven approaches, which
are widely adopted but face challenges including dataset bias, overfitting, and …

Embodied multi-modal agent trained by an llm from a parallel textworld

Y Yang, T Zhou, K Li, D Tao, L Li… - Proceedings of the …, 2024 - openaccess.thecvf.com
While large language models (LLMs) excel in a simulated world of texts they struggle to
interact with the more realistic world without perceptions of other modalities such as visual or …

Large language models empowered agent-based modeling and simulation: A survey and perspectives

C Gao, X Lan, N Li, Y Yuan, J Ding, Z Zhou… - arXiv preprint arXiv …, 2023 - arxiv.org
Agent-based modeling and simulation has evolved as a powerful tool for modeling complex
systems, offering insights into emergent behaviors and interactions among diverse agents …

Vision language models in autonomous driving and intelligent transportation systems

X Zhou, M Liu, BL Zagar, E Yurtsever… - arXiv preprint arXiv …, 2023 - arxiv.org
The applications of Vision-Language Models (VLMs) in the fields of Autonomous Driving
(AD) and Intelligent Transportation Systems (ITS) have attracted widespread attention due to …

On the road with gpt-4v (ision): Early explorations of visual-language model on autonomous driving

L Wen, X Yang, D Fu, X Wang, P Cai, X Li, T Ma… - arXiv preprint arXiv …, 2023 - arxiv.org
The pursuit of autonomous driving technology hinges on the sophisticated integration of
perception, decision-making, and control systems. Traditional approaches, both data-driven …

Chatgpt as your vehicle co-pilot: An initial attempt

S Wang, Y Zhu, Z Li, Y Wang, L Li… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
One of the most challenging problems in human-machine co-work is the gap between
human intention and the machine's understanding and execution. Large Language Models …