Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.04.2024 · 20 (4) · 1-20
DOI: 10.1145/3633517 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.01.2024. Verfasst von Yijie Hu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 55 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Jiangsu Science and Technology Programme, European Union’s Horizon 2020 research and innovation programme.
Abstract (Originalsprache)
Scene text recognition (STR), one typical sequence-to-sequence problem, has drawn much attention recently in multimedia applications. To guarantee good performance, it is essential for STR to obtain aligned character-wise features from the whole-image feature maps. While most present works adopt fully data-driven attention-based alignment, such practice ignores specific character geometric information. In this article, built upon a group of learnable geometric points, we propose a novel shape-driven attention alignment method that is able to obtain character-wise features. Concretely, we first design a corner detector to generate a shape map to guide the attention alignments explicitly, where a series of points can be learned to represent character-wise features flexibly. We then propose a dual-path network with a mutual learning and cooperating strategy that successfully combines CNN with a ViT-based model, leading to further accuracy improvement. We conduct extensive experiments to evaluate the proposed method on various scene text benchmarks, including six popular regular and irregular datasets, two more challenging datasets (i.e., WordArt and OST), and three Chinese datasets. Experimental results indicate that our method can achieve superior performance with a comparable model size against many state-of-the-art models.
Förderung
- National Natural Science Foundation of China
- Jiangsu Science and Technology Programme
- European Union’s Horizon 2020 research and innovation programme
- UK EPSRC under projects
Zitieren
APA
Hu, Y., Dong, B., Huang, K., Ding, L., Wang, W., Huang, X., & Wang, Q. (2024). Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(4), 1-20. https://doi.org/10.1145/3633517
Vancouver
Hu Y, Dong B, Huang K, Ding L, Wang W, Huang X, et al. Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(4):1-20. doi:10.1145/3633517
BibTeX
@article{Hu2024,
author = {Hu, Yijie and Dong, Bin and Huang, Kaizhu and Ding, Lei and Wang, Wei and Huang, Xiaowei and Wang, Qiu-Feng},
title = {Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2024},
volume = {20},
number = {4},
pages = {1--20},
doi = {10.1145/3633517}
}
EUVIMED