ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (7) · 7396-7404
DOI: 10.1609/aaai.v38i7.28570 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Zhanjie Zhang und 9 weiteren Autor:innen. Die Arbeit wurde bisher 66-mal zitiert (Crossref).
Abstract (Originalsprache)
Artistic style transfer aims to repaint the content image with the learned artistic style. Existing artistic style transfer methods can be divided into two categories: small model-based approaches and pre-trained large-scale model-based approaches. Small model-based approaches can preserve the content strucuture, but fail to produce highly realistic stylized images and introduce artifacts and disharmonious patterns; Pre-trained large-scale model-based approaches can generate highly realistic stylized images but struggle with preserving the content structure. To address the above issues, we propose ArtBank, a novel artistic style transfer framework, to generate highly realistic stylized images while preserving the content structure of the content images. Specifically, to sufficiently dig out the knowledge embedded in pre-trained large-scale models, an Implicit Style Prompt Bank (ISPB), a set of trainable parameter matrices, is designed to learn and store knowledge from the collection of artworks and behave as a visual prompt to guide pre-trained large-scale models to generate highly realistic stylized images while preserving content structure. Besides, to accelerate training the above ISPB, we propose a novel Spatial-Statistical-based self-Attention Module (SSAM). The qualitative and quantitative experiments demonstrate the superiority of our proposed method over state-of-the-art artistic style transfer methods. Code is available at https://github.com/Jamie-Cheung/ArtBank.
Zitieren
APA
Zhang, Z., Zhang, Q., Xing, W., Li, G., Zhao, L., Sun, J., Lan, Z., Luan, J., Huang, Y., & Lin, H. (2024). ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank. Proceedings of the AAAI Conference on Artificial Intelligence, 38(7), 7396-7404. https://doi.org/10.1609/aaai.v38i7.28570
Vancouver
Zhang Z, Zhang Q, Xing W, Li G, Zhao L, Sun J, et al. ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(7):7396-7404. doi:10.1609/aaai.v38i7.28570
BibTeX
@article{Zhang2024,
author = {Zhang, Zhanjie and Zhang, Quanwei and Xing, Wei and Li, Guangyuan and Zhao, Lei and Sun, Jiakai and Lan, Zehua and Luan, Junsheng and Huang, Yiling and Lin, Huaizhong},
title = {ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {7},
pages = {7396--7404},
doi = {10.1609/aaai.v38i7.28570}
}
EUVIMED