PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (6) · 4412-4420
DOI: 10.1609/aaai.v40i6.42439 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Qi Guo und 7 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).
Abstract (Originalsprache)
Multimodal Large Language Models (MLLMs) are becoming integral to autonomous driving (AD) systems due to their strong vision-language reasoning capabilities. However, MLLMs are vulnerable to adversarial attacks—particularly adversarial patch attacks—which can pose serious threats in real-world scenarios. Existing patch-based attack methods are primarily designed for object detection models. Due to the more complex architectures and strong reasoning capabilities of MLLMs, these approaches perform poorly when transferred to MLLM-based systems. To address these limitations, we propose PhysPatch, a physically realizable and transferable adversarial patch framework tailored for MLLM-based AD systems. PhysPatch jointly optimizes patch location, shape, and content to enhance attack effectiveness and real-world applicability. It introduces a semantic-based mask initialization strategy for realistic placement, an SVD-based local alignment loss with patch-guided crop-resize to improve transferability, and a potential field-based mask refinement method. Extensive experiments across open-source, commercial, and reasoning-capable MLLMs demonstrate that PhysPatch significantly outperforms state-of-the-art (SOTA) methods in steering MLLM-based AD systems toward target-aligned perception and planning outputs. Moreover, PhysPatch consistently places adversarial patches in physically feasible regions of AD scenes, ensuring strong real-world applicability and deployability.
Zitieren
APA
Guo, Q., Jia, X., Pang, S., Qin, S., Wang, L., Jia, J., Liu, Y., & Guo, Q. (2026). PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems. Proceedings of the AAAI Conference on Artificial Intelligence, 40(6), 4412-4420. https://doi.org/10.1609/aaai.v40i6.42439
Vancouver
Guo Q, Jia X, Pang S, Qin S, Wang L, Jia J, et al. PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(6):4412-4420. doi:10.1609/aaai.v40i6.42439
BibTeX
@article{Guo2026,
author = {Guo, Qi and Jia, Xiaojun and Pang, Shanmin and Qin, Simeng and Wang, Lin and Jia, Ju and Liu, Yang and Guo, Qing},
title = {PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {6},
pages = {4412--4420},
doi = {10.1609/aaai.v40i6.42439}
}
EUVIMED