EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Image

Dani Valevski, Matan Kalman, Eyal Molad, Eyal Segalis, Yossi Matias, Yaniv Leviathan

ACM Transactions on Graphics · 2023-08 · 42 (4) · 1-10

DOI: 10.1145/3592451 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 26.07.2023. Verfasst von Dani Valevski und 5 weiteren Autor:innen. Die Arbeit wurde bisher 69-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-driven image generation methods have shown impressive results recently, allowing casual users to generate high quality images by providing textual descriptions. However, similar capabilities for editing existing images are still out of reach. Text-driven image editing methods usually need edit masks, struggle with edits that require significant visual changes and cannot easily keep specific details of the edited portion. In this paper we make the observation that image-generation models can be converted to image-editing models simply by fine-tuning them on a single image. We also show that initializing the stochastic sampler with a noised version of the base image before the sampling and interpolating relevant details from the base image after sampling further increase the quality of the edit operation. Combining these observations, we propose UniTune, a novel image editing method. UniTune gets as input an arbitrary image and a textual edit description, and carries out the edit while maintaining high fidelity to the input image. UniTune does not require additional inputs, like masks or sketches, and can perform multiple edits on the same image without retraining. We test our method using the Imagen model in a range of different use cases. We demonstrate that it is broadly applicable and can perform a surprisingly wide range of expressive editing operations, including those requiring significant visual changes that were previously impossible.

Zitieren

APA

Valevski, D., Kalman, M., Molad, E., Segalis, E., Matias, Y., & Leviathan, Y. (2023). UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Image. ACM Transactions on Graphics, 42(4), 1-10. https://doi.org/10.1145/3592451

Vancouver

Valevski D, Kalman M, Molad E, Segalis E, Matias Y, Leviathan Y. UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Image. ACM Transactions on Graphics. 2023;42(4):1-10. doi:10.1145/3592451

BibTeX
@article{Valevski2023,
  author = {Valevski, Dani and Kalman, Matan and Molad, Eyal and Segalis, Eyal and Matias, Yossi and Leviathan, Yaniv},
  title = {UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Image},
  journal = {ACM Transactions on Graphics},
  year = {2023},
  volume = {42},
  number = {4},
  pages = {1--10},
  doi = {10.1145/3592451}
}
Kostenlos starten →