EUVIMEDEUVIMED entdecken
Crossref

All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models

SeungHoo Hong, Juhun Lee, Simon S. Woo

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (19) · 21143-21151

DOI: 10.1609/aaai.v38i19.30107 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von SeungHoo Hong und 2 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-to-Image models such as Stable Diffusion have shown impressive image generation synthesis, thanks to the utilization of large-scale datasets. However, these datasets may contain sexually explicit, copyrighted, or undesirable content, which allows the model to directly generate them. Given that retraining these large models on individual concept deletion requests is infeasible, fine-tuning algorithms have been developed to tackle concept erasing in diffusion models. While these algorithms yield good concept erasure, they all present one of the following issues: 1) the corrupted feature space yields synthesis of disintegrated objects, 2) the initially synthesized content undergoes a divergence in both spatial structure and semantics in the generated images, and 3) sub-optimal training updates heighten the model's susceptibility to utility harm. These issues severely degrade the original utility of generative models. In this work, we present a new approach that solves all of these challenges. We take inspiration from the concept of classifier guidance and propose a surgical update on the classifier guidance term while constraining the drift of the unconditional score term. Furthermore, our algorithm empowers the user to select an alternative to the erasing concept, allowing for more controllability. Our experimental results show that our algorithm not only erases the target concept effectively but also preserves the model’s generation capability.

Zitieren

APA

Hong, S., Lee, J., & Woo, S. S. (2024). All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(19), 21143-21151. https://doi.org/10.1609/aaai.v38i19.30107

Vancouver

Hong S, Lee J, Woo SS. All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(19):21143-21151. doi:10.1609/aaai.v38i19.30107

BibTeX
@article{Hong2024,
  author = {Hong, SeungHoo and Lee, Juhun and Woo, Simon S.},
  title = {All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {19},
  pages = {21143--21151},
  doi = {10.1609/aaai.v38i19.30107}
}
Kostenlos starten →