EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Training-Free Consistent Text-to-Image Generation

Yoad Tewel, Omri Kaduri, Rinon Gal, Yoni Kasten, Lior Wolf, Gal Chechik, Yuval Atzmon

ACM Transactions on Graphics · 19.07.2024 · 43 (4) · 1-18

DOI: 10.1145/3658157 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 19.07.2024. Verfasst von Yoad Tewel und 6 weiteren Autor:innen. Die Arbeit wurde bisher 86-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-to-image models offer a new level of creative flexibility by allowing users to guide the image generation process through natural language. However, using these models to consistently portray the same subject across diverse prompts remains challenging. Existing approaches fine-tune the model to teach it new words that describe specific user-provided subjects or add image conditioning to the model. These methods require lengthy persubject optimization or large-scale pre-training. Moreover, they struggle to align generated images with text prompts and face difficulties in portraying multiple subjects. Here, we present ConsiStory , a training-free approach that enables consistent subject generation by sharing the internal activations of the pretrained model. We introduce a subject-driven shared attention block and correspondence-based feature injection to promote subject consistency between images. Additionally, we develop strategies to encourage layout diversity while maintaining subject consistency. We compare ConsiStory to a range of baselines, and demonstrate state-of-the-art performance on subject consistency and text alignment, without requiring a single optimization step. Finally, ConsiStory can naturally extend to multi-subject scenarios, and even enable training-free personalization for common objects.

Zitieren

APA

Tewel, Y., Kaduri, O., Gal, R., Kasten, Y., Wolf, L., Chechik, G., & Atzmon, Y. (2024). Training-Free Consistent Text-to-Image Generation. ACM Transactions on Graphics, 43(4), 1-18. https://doi.org/10.1145/3658157

Vancouver

Tewel Y, Kaduri O, Gal R, Kasten Y, Wolf L, Chechik G, et al. Training-Free Consistent Text-to-Image Generation. ACM Transactions on Graphics. 2024;43(4):1-18. doi:10.1145/3658157

BibTeX
@article{Tewel2024,
  author = {Tewel, Yoad and Kaduri, Omri and Gal, Rinon and Kasten, Yoni and Wolf, Lior and Chechik, Gal and Atzmon, Yuval},
  title = {Training-Free Consistent Text-to-Image Generation},
  journal = {ACM Transactions on Graphics},
  year = {2024},
  volume = {43},
  number = {4},
  pages = {1--18},
  doi = {10.1145/3658157}
}
Kostenlos starten →