T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (42) · 36039-36047
DOI: 10.1609/aaai.v40i42.40920 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Chenyu Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref).
Abstract (Originalsprache)
Using risky text prompts, such as pornography and violent prompts, to test the safety of text-to-image (T2I) models is a critical task. However, existing risky prompt datasets are limited in three key areas: 1) limited risky categories, 2) coarse-grained annotation, and 3) low effectiveness. To address these limitations, we introduce T2I-RiskyPrompt, a comprehensive benchmark designed for evaluating safety-related tasks in T2I models. Specifically, we first develop a hierarchical risk taxonomy, which consists of 6 primary categories and 14 fine-grained subcategories. Building upon this taxonomy, we construct a pipeline to collect and annotate risky prompts. Finally, we obtain 6,432 effective risky prompts, where each prompt is annotated with both hierarchical category labels and detailed risk reasons. Moreover, to facilitate the evaluation, we propose a reason-driven risky image detection method that explicitly aligns the MLLM with safety annotations. Based on T2I-RiskyPrompt, we conduct a comprehensive evaluation of eight T2I models, nine defense methods, five safety filters, and five attack strategies, offering nine key insights into the strengths and limitations of T2I model safety. Finally, we discuss potential applications of T2I-RiskyPrompt across various research fields.
Zitieren
APA
Zhang, C., Zhang, T., Wang, L., Chen, R., Li, W., & Liu, A. (2026). T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model. Proceedings of the AAAI Conference on Artificial Intelligence, 40(42), 36039-36047. https://doi.org/10.1609/aaai.v40i42.40920
Vancouver
Zhang C, Zhang T, Wang L, Chen R, Li W, Liu A. T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(42):36039-36047. doi:10.1609/aaai.v40i42.40920
BibTeX
@article{Zhang2026,
author = {Zhang, Chenyu and Zhang, Tairen and Wang, Lanjun and Chen, Ruidong and Li, Wenhui and Liu, Anan},
title = {T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Model},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {42},
pages = {36039--36047},
doi = {10.1609/aaai.v40i42.40920}
}
EUVIMED