Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (7) · 7215-7223
DOI: 10.1609/aaai.v38i7.28550 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Lingjun Zhang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 30-mal zitiert (Crossref).
Abstract (Originalsprache)
Recently, diffusion-based image generation methods are credited for their remarkable text-to-image generation capabilities, while still facing challenges in accurately generating multilingual scene text images. To tackle this problem, we propose Diff-Text, which is a training-free scene text generation framework for any language. Our model outputs a photo-realistic image given a text of any language along with a textual description of a scene. The model leverages rendered sketch images as priors, thus arousing the potential multilingual-generation ability of the pre-trained Stable Diffusion. Based on the observation from the influence of the cross-attention map on object placement in generated images, we propose a localized attention constraint into the cross-attention layer to address the unreasonable positioning problem of scene text. Additionally, we introduce contrastive image-level prompts to further refine the position of the textual region and achieve more accurate scene text generation. Experiments demonstrate that our method outperforms the existing method in both the accuracy of text recognition and the naturalness of foreground-background blending.
Zitieren
APA
Zhang, L., Chen, X., Wang, Y., Lu, Y., & Qiao, Y. (2024). Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model. Proceedings of the AAAI Conference on Artificial Intelligence, 38(7), 7215-7223. https://doi.org/10.1609/aaai.v38i7.28550
Vancouver
Zhang L, Chen X, Wang Y, Lu Y, Qiao Y. Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(7):7215-7223. doi:10.1609/aaai.v38i7.28550
BibTeX
@article{Zhang2024,
author = {Zhang, Lingjun and Chen, Xinyuan and Wang, Yaohui and Lu, Yue and Qiao, Yu},
title = {Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {7},
pages = {7215--7223},
doi = {10.1609/aaai.v38i7.28550}
}
EUVIMED