PersGuard: Preventing Malicious Personalization in Text-to-Image Diffusion Models via Model Backdoors
IEEE Transactions on Dependable and Secure Computing · 2026-07 · 23 (4) · 9682-9694
DOI: 10.1109/tdsc.2026.3694473 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Dependable and Secure Computing, 2026-07. Verfasst von Xinwei Liu und 4 weiteren Autor:innen. Die Arbeit nennt 54 Literaturangaben. Gefördert von: Shenzhen Science and Technology Program, National Natural Science Foundation of China, Open Topics from the Lion Rock Labs of Cyberspace Security.
Abstract (Originalsprache)
Diffusion models (DMs) have advanced text-to-image (T2I) synthesis, yet their personalization capabilities raise serious privacy and copyright concerns. Malicious actors can misuse these models to generate unauthorized portraits or artistic style replicas. Existing proactive defenses primarily rely on applying adversarial perturbations to reference images to disrupt training. However, these approaches face limitations: they assume all training images are pre-perturbed and are prone to failure when datasets contain unperturbed images or undergo minor data transformations. In this paper, we introduce PersGuard, a novel backdoor-based framework designed to prevent unauthorized personalization of pre-trained T2I diffusion models. Unlike perturbation-based methods, we assume protectors can embed protective backdoors into the models before their release. This mechanism ensures that if a downstream user fine-tunes the model on protected images, the model retains the backdoor and generates predefined protective outputs; conversely, for unprotected images, the backdoor is effectively removed during fine-tuning to ensure normal model utility. We formulate the backdoor injection as a unified optimization problem incorporating three objectives: a backdoor behavior loss to activate protection, a prior preservation loss to maintain standard generation capabilities, and a novel backdoor retention loss. The retention loss is specifically designed to mirror personalization loss, ensuring the backdoor remains robust during downstream fine-tuning. Extensive experiments across gray-box and black-box settings, multi-object protection, and facial identity protection demonstrate that PersGuard provides superior privacy protection compared to existing perturbation-based methods.
Förderung
- Shenzhen Science and Technology Program
- National Natural Science Foundation of China
- Open Topics from the Lion Rock Labs of Cyberspace Security
Zitieren
APA
Liu, X., Jia, X., Xun, Y., Zhang, H., & Cao, X. (2026). PersGuard: Preventing Malicious Personalization in Text-to-Image Diffusion Models via Model Backdoors. IEEE Transactions on Dependable and Secure Computing, 23(4), 9682-9694. https://doi.org/10.1109/tdsc.2026.3694473
Vancouver
Liu X, Jia X, Xun Y, Zhang H, Cao X. PersGuard: Preventing Malicious Personalization in Text-to-Image Diffusion Models via Model Backdoors. IEEE Transactions on Dependable and Secure Computing. 2026;23(4):9682-9694. doi:10.1109/tdsc.2026.3694473
BibTeX
@article{Liu2026,
author = {Liu, Xinwei and Jia, Xiaojun and Xun, Yuan and Zhang, Hua and Cao, Xiaochun},
title = {PersGuard: Preventing Malicious Personalization in Text-to-Image Diffusion Models via Model Backdoors},
journal = {IEEE Transactions on Dependable and Secure Computing},
year = {2026},
volume = {23},
number = {4},
pages = {9682--9694},
doi = {10.1109/tdsc.2026.3694473}
}
EUVIMED