EUVIMEDEUVIMED entdecken
Crossref

Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuning

Ankita Raj, Chetan Arora

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (44) · 37849-37857

DOI: 10.1609/aaai.v40i44.41121 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Ankita Raj und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Open-vocabulary object detectors (OVODs) unify vision and language to detect arbitrary object categories based on text prompts, enabling strong zero-shot generalization to novel concepts. As these models gain traction in high-stakes applications such as robotics, autonomous driving, and surveillance, understanding their security risks becomes crucial. In this work, we conduct the first study of backdoor attacks on OVODs and reveal a new attack surface introduced by prompt tuning. We propose TrAP (Trigger-Aware Prompt tuning), a multi-modal backdoor injection strategy that jointly optimizes prompt parameters in both image and text modalities along with visual triggers. TrAP enables the attacker to implant malicious behavior using lightweight, learnable prompt tokens without retraining the base model weights, thus preserving generalization while embedding a hidden backdoor. We adopt a curriculum-based training strategy that progressively shrinks the trigger size, enabling effective backdoor activation using small trigger patches at inference. Experiments across multiple datasets show that TrAP achieves high attack success rates for both object misclassification and object disappearance attacks, while also improving clean image performance on downstream datasets compared to the zero-shot setting.

Zitieren

APA

Raj, A., & Arora, C. (2026). Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(44), 37849-37857. https://doi.org/10.1609/aaai.v40i44.41121

Vancouver

Raj A, Arora C. Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(44):37849-37857. doi:10.1609/aaai.v40i44.41121

BibTeX
@article{Raj2026,
  author = {Raj, Ankita and Arora, Chetan},
  title = {Backdoor Attacks on Open Vocabulary Object Detectors via Multi-Modal Prompt Tuning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {44},
  pages = {37849--37857},
  doi = {10.1609/aaai.v40i44.41121}
}
Kostenlos starten →