EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

QG-STR: Training-Time Optimized Question-Guided Scene Text Recognition via Visual Question Answering

Quanxing Xu, Ling Zhou, Xian Zhong, Feifei Zhang, Rubing Huang

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2026 · 22 (8) · 1-20

DOI: 10.1145/3822602 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 10.08.2026. Verfasst von Quanxing Xu und 4 weiteren Autor:innen. Die Arbeit nennt 73 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Scene Text Spotting (STS) aims to transcribe text embedded in natural images, typically encompassing Scene Text Detection (STD) and Scene Text Recognition (STR) . Advances in image understanding have made end-to-end text spotting increasingly viable. Concurrently, multimodal research has highlighted the potential of vision-language reasoning tasks, such as Visual Question Answering (VQA) . To leverage multimodal reasoning for STR, we propose a training-time question-guided STR framework that integrates VQA, termed Question-Guided STR (QG-STR) . The framework unifies STR, Visual Question Generation (VQG) , and VQA within a single architecture, enabling multimodal reasoning to enhance text-spotting performance. Specifically, visual understanding and logical reasoning are used as supervisory signals during training to improve text recognition accuracy and boost end-to-end text spotting. QG-STR is model-agnostic and compatible with diverse STR and VQA architectures, employing question guidance solely as a training-time supervision mechanism. During inference, the STR module functions independently without requiring external questions. Extensive experiments on Total-Text , ICDAR2015 , ICDAR2013 , and CTW1500 validate the effectiveness of QG-STR.

Zitieren

APA

Xu, Q., Zhou, L., Zhong, X., Zhang, F., & Huang, R. (2026). QG-STR: Training-Time Optimized Question-Guided Scene Text Recognition via Visual Question Answering. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(8), 1-20. https://doi.org/10.1145/3822602

Vancouver

Xu Q, Zhou L, Zhong X, Zhang F, Huang R. QG-STR: Training-Time Optimized Question-Guided Scene Text Recognition via Visual Question Answering. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(8):1-20. doi:10.1145/3822602

BibTeX
@article{Xu2026,
  author = {Xu, Quanxing and Zhou, Ling and Zhong, Xian and Zhang, Feifei and Huang, Rubing},
  title = {QG-STR: Training-Time Optimized Question-Guided Scene Text Recognition via Visual Question Answering},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {8},
  pages = {1--20},
  doi = {10.1145/3822602}
}
Kostenlos starten →