Physical Backdoor Attack can Jeopardize Driving with Vision-Large-Language Models

Z Ni, R Ye, Y Wei, Z Xiang, Y Wang, S Chen - arXiv preprint arXiv …, 2024 - arxiv.org
Vision-Large-Language-models (VLMs) have great application prospects in autonomous
driving. Despite the ability of VLMs to comprehend and make decisions in complex …

Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving with Typography

N Chung, S Gao, TA Vu, J Zhang, A Liu, Y Lin… - arXiv preprint arXiv …, 2024 - arxiv.org
Vision-Large-Language-Models (Vision-LLMs) are increasingly being integrated into
autonomous driving (AD) systems due to their advanced visual-language reasoning …

Targeted attack on deep rl-based autonomous driving with learned visual patterns

P Buddareddygari, T Zhang, Y Yang… - … on Robotics and …, 2022 - ieeexplore.ieee.org
Recent studies demonstrated the vulnerability of control policies learned through deep
reinforcement learning against adversarial attacks, raising concerns about the application of …

Physical backdoor attacks to lane detection systems in autonomous driving

X Han, G Xu, Y Zhou, X Yang, J Li… - Proceedings of the 30th …, 2022 - dl.acm.org
Modern autonomous vehicles adopt state-of-the-art DNN models to interpret the sensor data
and perceive the environment. However, DNN models are vulnerable to different types of …

Figstep: Jailbreaking large vision-language models via typographic visual prompts

Y Gong, D Ran, J Liu, C Wang, T Cong, A Wang… - arXiv preprint arXiv …, 2023 - arxiv.org
Large vision-language models (VLMs) like GPT-4V represent an unprecedented revolution
in the field of artificial intelligence (AI). Compared to single-modal large language models …

Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

Y Zong, O Bohdal, T Yu, Y Yang… - arXiv preprint arXiv …, 2024 - arxiv.org
Current vision large language models (VLLMs) exhibit remarkable capabilities yet are prone
to generate harmful content and are vulnerable to even the simplest jailbreaking attacks. Our …

How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments?

C Wen, J Liang, S Yuan, H Huang, Y Fang - arXiv preprint arXiv …, 2024 - arxiv.org
In the field of robotics and automation, navigation systems based on Large Language
Models (LLMs) have recently shown impressive performance. However, the security aspects …

Attacking vision-based perception in end-to-end autonomous driving models

A Boloor, K Garimella, X He, C Gill… - Journal of Systems …, 2020 - Elsevier
Recent advances in machine learning, especially techniques such as deep neural networks,
are enabling a range of emerging applications. One such example is autonomous driving …

Safety Alignment for Vision Language Models

Z Liu, Y Nie, Y Tan, X Yue, Q Cui, C Wang… - arXiv preprint arXiv …, 2024 - arxiv.org
Benefiting from the powerful capabilities of Large Language Models (LLMs), pre-trained
visual encoder models connected to an LLMs can realize Vision Language Models (VLMs) …

Towards Robust Physical-world Backdoor Attacks on Lane Detection

X Zhang, A Liu, T Zhang, S Liang, X Liu - arXiv preprint arXiv:2405.05553, 2024 - arxiv.org
Deep learning-based lane detection (LD) plays a critical role in autonomous driving
systems, such as adaptive cruise control. However, it is vulnerable to backdoor attacks …