EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Fast and Lightweight Vision-Language Model for Adversarial Traffic Sign Detection

Furkan Mumcu, Yasin Yilmaz

Electronics · 2024-06-03 · 13 (11) · 2172

DOI: 10.3390/electronics13112172 ↗

At a glance

Journal article in Electronics, published online on 2024-06-03. Written by Furkan Mumcu and 1 co-authors. Cited 9 times so far (Crossref); 47 references. The full text is freely available (open access, CC BY 4.0). Funded by: U.S. National Science Foundation (NSF).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Several attacks have been proposed against autonomous vehicles and their subsystems that are powered by machine learning (ML). Road sign recognition models are especially heavily tested under various adversarial ML attack settings, and they have proven to be vulnerable. Despite the increasing research on adversarial ML attacks against road sign recognition models, there is little to no focus on defending against these attacks. In this paper, we propose the first defense method specifically designed for autonomous vehicles to detect adversarial ML attacks targeting road sign recognition models, which is called ViLAS (Vision-Language Model for Adversarial Traffic Sign Detection). The proposed defense method is based on a custom, fast, lightweight, and salable vision-language model (VLM) and is compatible with any existing traffic sign recognition system. Thanks to the orthogonal information coming from the class label text data through the language model, ViLAS leverages image context in addition to visual data for highly effective attack detection performance. In our extensive experiments, we show that our method consistently detects various attacks against different target models with high true positive rates while satisfying very low false positive rates. When tested against four state-of-the-art attacks targeting four popular action recognition models, our proposed detector achieves an average AUC of 0.94. This result achieves a 25.3% improvement over a state-of-the-art defense method proposed for generic image attack detection, which attains an average AUC of 0.75. We also show that our custom VLM is more suitable for an autonomous vehicle compared to the popular off-the-shelf VLM and CLIP in terms of speed (4.4 vs. 9.3 milliseconds), space complexity (0.36 vs. 1.6 GB), and performance (0.94 vs. 0.43 average AUC).

Funding

  • U.S. National Science Foundation (NSF)

Cite

APA

Mumcu, F., & Yilmaz, Y. (2024). Fast and Lightweight Vision-Language Model for Adversarial Traffic Sign Detection. Electronics, 13(11), 2172. https://doi.org/10.3390/electronics13112172

Vancouver

Mumcu F, Yilmaz Y. Fast and Lightweight Vision-Language Model for Adversarial Traffic Sign Detection. Electronics. 2024;13(11):2172. doi:10.3390/electronics13112172

BibTeX
@article{Mumcu2024,
  author = {Mumcu, Furkan and Yilmaz, Yasin},
  title = {Fast and Lightweight Vision-Language Model for Adversarial Traffic Sign Detection},
  journal = {Electronics},
  year = {2024},
  volume = {13},
  number = {11},
  pages = {2172},
  doi = {10.3390/electronics13112172}
}
Start for free →