EUVIMEDEUVIMED entdecken
Crossref

Vulnerability-Aware Robust Multimodal Adversarial Training

Junrui Zhang, Xinyu Zhao, Jie Peng, Chenjie Wang, Jianmin Ji, Tianlong Chen

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (33) · 28265-28273

DOI: 10.1609/aaai.v40i33.40054 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Junrui Zhang und 5 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal learning has shown significant superiority on various tasks by integrating multiple modalities. However, the interdependencies among modalities increase the susceptibility of multimodal models to adversarial attacks. Existing methods mainly focus on attacks on specific modalities or indiscriminately attack all modalities. In this paper, we find that these approaches ignore the differences between modalities in their contribution to final robustness, resulting in suboptimal robustness performance. To bridge this gap, we introduce Vulnerability-Aware Robust Multimodal Adversarial Training (VARMAT), a probe-in-training adversarial training method that improves multimodal robustness by identifying the vulnerability of each modality. To be specific, VARMAT first explicitly quantifies the vulnerability of each modality, grounded in a first-order approximation of the attack objective (Probe). Then, we propose a targeted regularization term that penalizes modalities with high vulnerability, guiding robust learning while maintaining task accuracy (Training). We demonstrate the enhanced robustness of our method across multiple multimodal datasets involving diverse modalities. Finally, we achieve {12.73%, 22.21%, 11.19%} robustness improvement on three multimodal datasets, revealing a significant blind spot in multimodal adversarial training.

Zitieren

APA

Zhang, J., Zhao, X., Peng, J., Wang, C., Ji, J., & Chen, T. (2026). Vulnerability-Aware Robust Multimodal Adversarial Training. Proceedings of the AAAI Conference on Artificial Intelligence, 40(33), 28265-28273. https://doi.org/10.1609/aaai.v40i33.40054

Vancouver

Zhang J, Zhao X, Peng J, Wang C, Ji J, Chen T. Vulnerability-Aware Robust Multimodal Adversarial Training. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(33):28265-28273. doi:10.1609/aaai.v40i33.40054

BibTeX
@article{Zhang2026,
  author = {Zhang, Junrui and Zhao, Xinyu and Peng, Jie and Wang, Chenjie and Ji, Jianmin and Chen, Tianlong},
  title = {Vulnerability-Aware Robust Multimodal Adversarial Training},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {33},
  pages = {28265--28273},
  doi = {10.1609/aaai.v40i33.40054}
}
Kostenlos starten →