Decoupled Textual Embeddings for Customized Image Generation
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (2) · 909-917
DOI: 10.1609/aaai.v38i2.27850 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yufei Cai und 5 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref).
Abstract (Originalsprache)
Customized text-to-image generation, which aims to learn user-specified concepts with a few images, has drawn significant attention recently. However, existing methods usually suffer from overfitting issues and entangle the subject-unrelated information (e.g., background and pose) with the learned concept, limiting the potential to compose concept into new scenes. To address these issues, we propose the DETEX, a novel approach that learns the disentangled concept embedding for flexible customized text-to-image generation. Unlike conventional methods that learn a single concept embedding from the given images, our DETEX represents each image using multiple word embeddings during training, i.e., a learnable image-shared subject embedding and several image-specific subject-unrelated embeddings. To decouple irrelevant attributes (i.e., background and pose) from the subject embedding, we further present several attribute mappers that encode each image as several image-specific subject-unrelated embeddings. To encourage these unrelated embeddings to capture the irrelevant information, we incorporate them with corresponding attribute words and propose a joint training strategy to facilitate the disentanglement. During inference, we only use the subject embedding for image generation, while selectively using image-specific embeddings to retain image-specified attributes. Extensive experiments demonstrate that the subject embedding obtained by our method can faithfully represent the target concept, while showing superior editability compared to the state-of-the-art methods. Our code will be available at https://github.com/PrototypeNx/DETEX.
Zitieren
APA
Cai, Y., Wei, Y., Ji, Z., Bai, J., Han, H., & Zuo, W. (2024). Decoupled Textual Embeddings for Customized Image Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(2), 909-917. https://doi.org/10.1609/aaai.v38i2.27850
Vancouver
Cai Y, Wei Y, Ji Z, Bai J, Han H, Zuo W. Decoupled Textual Embeddings for Customized Image Generation. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(2):909-917. doi:10.1609/aaai.v38i2.27850
BibTeX
@article{Cai2024,
author = {Cai, Yufei and Wei, Yuxiang and Ji, Zhilong and Bai, Jinfeng and Han, Hu and Zuo, Wangmeng},
title = {Decoupled Textual Embeddings for Customized Image Generation},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {2},
pages = {909--917},
doi = {10.1609/aaai.v38i2.27850}
}
EUVIMED