Text Image Inpainting via Global Structure-Guided Diffusion Models
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (7) · 7775-7783
DOI: 10.1609/aaai.v38i7.28612 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Shipeng Zhu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 31-mal zitiert (Crossref).
Abstract (Originalsprache)
Real-world text can be damaged by corrosion issues caused by environmental or human factors, which hinder the preservation of the complete styles of texts, e.g., texture and structure. These corrosion issues, such as graffiti signs and incomplete signatures, bring difficulties in understanding the texts, thereby posing significant challenges to downstream applications, e.g., scene text recognition and signature identification. Notably, current inpainting techniques often fail to adequately address this problem and have difficulties restoring accurate text images along with reasonable and consistent styles. Formulating this as an open problem of text image inpainting, this paper aims to build a benchmark to facilitate its study. In doing so, we establish two specific text inpainting datasets which contain scene text images and handwritten text images, respectively. Each of them includes images revamped by real-life and synthetic datasets, featuring pairs of original images, corrupted images, and other assistant information. On top of the datasets, we further develop a novel neural framework, Global Structure-guided Diffusion Model (GSDM), as a potential solution. Leveraging the global structure of the text as a prior, the proposed GSDM develops an efficient diffusion model to recover clean texts. The efficacy of our approach is demonstrated by thorough empirical study, including a substantial boost in both recognition accuracy and image quality. These findings not only highlight the effectiveness of our method but also underscore its potential to enhance the broader field of text image understanding and processing. Code and datasets are available at: https://github.com/blackprotoss/GSDM.
Zitieren
APA
Zhu, S., Fang, P., Zhu, C., Zhao, Z., Xu, Q., & Xue, H. (2024). Text Image Inpainting via Global Structure-Guided Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(7), 7775-7783. https://doi.org/10.1609/aaai.v38i7.28612
Vancouver
Zhu S, Fang P, Zhu C, Zhao Z, Xu Q, Xue H. Text Image Inpainting via Global Structure-Guided Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(7):7775-7783. doi:10.1609/aaai.v38i7.28612
BibTeX
@article{Zhu2024,
author = {Zhu, Shipeng and Fang, Pengfei and Zhu, Chenjie and Zhao, Zuoyan and Xu, Qiang and Xue, Hui},
title = {Text Image Inpainting via Global Structure-Guided Diffusion Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {7},
pages = {7775--7783},
doi = {10.1609/aaai.v38i7.28612}
}
EUVIMED