EUVIMEDEUVIMED entdecken
Crossref

Decoupled Textual Embeddings for Customized Image Generation

Yufei Cai, Yuxiang Wei, Zhilong Ji, Jinfeng Bai, Hu Han, Wangmeng Zuo

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (2) · 909-917

DOI: 10.1609/aaai.v38i2.27850 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yufei Cai und 5 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Customized text-to-image generation, which aims to learn user-specified concepts with a few images, has drawn significant attention recently. However, existing methods usually suffer from overfitting issues and entangle the subject-unrelated information (e.g., background and pose) with the learned concept, limiting the potential to compose concept into new scenes. To address these issues, we propose the DETEX, a novel approach that learns the disentangled concept embedding for flexible customized text-to-image generation. Unlike conventional methods that learn a single concept embedding from the given images, our DETEX represents each image using multiple word embeddings during training, i.e., a learnable image-shared subject embedding and several image-specific subject-unrelated embeddings. To decouple irrelevant attributes (i.e., background and pose) from the subject embedding, we further present several attribute mappers that encode each image as several image-specific subject-unrelated embeddings. To encourage these unrelated embeddings to capture the irrelevant information, we incorporate them with corresponding attribute words and propose a joint training strategy to facilitate the disentanglement. During inference, we only use the subject embedding for image generation, while selectively using image-specific embeddings to retain image-specified attributes. Extensive experiments demonstrate that the subject embedding obtained by our method can faithfully represent the target concept, while showing superior editability compared to the state-of-the-art methods. Our code will be available at https://github.com/PrototypeNx/DETEX.

Zitieren

APA

Cai, Y., Wei, Y., Ji, Z., Bai, J., Han, H., & Zuo, W. (2024). Decoupled Textual Embeddings for Customized Image Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(2), 909-917. https://doi.org/10.1609/aaai.v38i2.27850

Vancouver

Cai Y, Wei Y, Ji Z, Bai J, Han H, Zuo W. Decoupled Textual Embeddings for Customized Image Generation. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(2):909-917. doi:10.1609/aaai.v38i2.27850

BibTeX
@article{Cai2024,
  author = {Cai, Yufei and Wei, Yuxiang and Ji, Zhilong and Bai, Jinfeng and Han, Hu and Zuo, Wangmeng},
  title = {Decoupled Textual Embeddings for Customized Image Generation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {2},
  pages = {909--917},
  doi = {10.1609/aaai.v38i2.27850}
}
Kostenlos starten →