Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (15) · 16605-16613
DOI: 10.1609/aaai.v38i15.29599 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Zihao Yu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref).
Abstract (Originalsprache)
Due to the recent success of diffusion models, text-to-image generation is becoming increasingly popular and achieves a wide range of applications. Among them, text-to-image editing, or continuous text-to-image generation, attracts lots of attention and can potentially improve the quality of generated images. It's common to see that users may want to slightly edit the generated image by making minor modifications to their input textual descriptions for several rounds of diffusion inference. However, such an image editing process suffers from the low inference efficiency of many existing diffusion models even using GPU accelerators. To solve this problem, we introduce Fast Image Semantically Edit (FISEdit), a cached-enabled sparse diffusion model inference engine for efficient text-to-image editing. The key intuition behind our approach is to utilize the semantic mapping between the minor modifications on the input text and the affected regions on the output image. For each text editing step, FISEdit can 1) automatically identify the affected image regions and 2) utilize the cached unchanged regions' feature map to accelerate the inference process. For the former, we measure the differences between cached and ad hoc feature maps given the modified textual description, extract the region with significant differences, and capture the affected region by masks. For the latter, we develop an efficient sparse diffusion inference engine that only computes the feature maps for the affected region while reusing the cached statistics for the rest of the image. Finally, extensive empirical results show that FISEdit can be 3.4 times and 4.4 times faster than existing methods on NVIDIA TITAN RTX and A100 GPUs respectively, and even generates more satisfactory images.
Zitieren
APA
Yu, Z., Li, H., Fu, F., Miao, X., & Cui, B. (2024). Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15), 16605-16613. https://doi.org/10.1609/aaai.v38i15.29599
Vancouver
Yu Z, Li H, Fu F, Miao X, Cui B. Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(15):16605-16613. doi:10.1609/aaai.v38i15.29599
BibTeX
@article{Yu2024,
author = {Yu, Zihao and Li, Haoyang and Fu, Fangcheng and Miao, Xupeng and Cui, Bin},
title = {Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {15},
pages = {16605--16613},
doi = {10.1609/aaai.v38i15.29599}
}
EUVIMED