BlackboxBench: A Comprehensive Benchmark of Black-Box Adversarial Attacks
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-09 · 47 (9) · 7867-7885
DOI: 10.1109/tpami.2025.3574432 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-09. Verfasst von Meixi Zheng und 4 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 90 Literaturangaben. Gefördert von: Basic and Applied Basic Research Foundation of Guangdong Province, Shenzhen Science and Technology Program, Sub-topic of Key R&D Projects of the Ministry of Science and Technology.
Abstract (Originalsprache)
Adversarial examples are well-known tools to evaluate the vulnerability of deep neural networks (DNNs). Although lots of adversarial attack algorithms have been developed, it's still challenging in the practical scenario that the model's parameters and architectures are inaccessible to the attacker/evaluator, i.e., black-box adversarial attacks. Due to the practical importance, there has been rapid progress from recent algorithms, reflected by the quick increase in attack success rate and quick decrease in query numbers to the target model. However, there lacks thorough evaluations and comparisons among these algorithms, causing difficulties in tracking the real progress, analyzing advantages and disadvantages of different technical routes, as well as designing future development roadmap of this field. Thus, we aim at building a comprehensive benchmark of black-box adversarial attacks, called BlackboxBench. It mainly provides: 1) a unified, extensible and modular-based codebase, implementing 29 query-based attack algorithms and 30 transfer-based attack algorithms; 2) comprehensive evaluations: we evaluate the implemented algorithms against several mainstreaming model architectures on 2 widely used datasets (CIFAR-10 and a subset of ImageNet), leading to 14,950 evaluations$^{1}$1 in total; 3) thorough analysis and new insights, as well analytical tools.
Förderung
- Basic and Applied Basic Research Foundation of Guangdong Province
- Shenzhen Science and Technology Program
- Sub-topic of Key R&D Projects of the Ministry of Science and Technology
- Longgang District Key Laboratory of Intelligent Digital Economy Security
- Guangdong Provincial Program
Zitieren
APA
Zheng, M., Yan, X., Zhu, Z., Chen, H., & Wu, B. (2025). BlackboxBench: A Comprehensive Benchmark of Black-Box Adversarial Attacks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(9), 7867-7885. https://doi.org/10.1109/tpami.2025.3574432
Vancouver
Zheng M, Yan X, Zhu Z, Chen H, Wu B. BlackboxBench: A Comprehensive Benchmark of Black-Box Adversarial Attacks. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(9):7867-7885. doi:10.1109/tpami.2025.3574432
BibTeX
@article{Zheng2025,
author = {Zheng, Meixi and Yan, Xuanchen and Zhu, Zihao and Chen, Hongrui and Wu, Baoyuan},
title = {BlackboxBench: A Comprehensive Benchmark of Black-Box Adversarial Attacks},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {9},
pages = {7867--7885},
doi = {10.1109/tpami.2025.3574432}
}
EUVIMED