EUVIMEDEUVIMED entdecken
Crossref

Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment

Yijie Hu, Bin Dong, Kaizhu Huang, Lei Ding, Wei Wang, Xiaowei Huang, Qiu-Feng Wang

ACM Transactions on Multimedia Computing, Communications, and Applications · 30.04.2024 · 20 (4) · 1-20

DOI: 10.1145/3633517 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 11.01.2024. Verfasst von Yijie Hu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 55 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Jiangsu Science and Technology Programme, European Union’s Horizon 2020 research and innovation programme.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Scene text recognition (STR), one typical sequence-to-sequence problem, has drawn much attention recently in multimedia applications. To guarantee good performance, it is essential for STR to obtain aligned character-wise features from the whole-image feature maps. While most present works adopt fully data-driven attention-based alignment, such practice ignores specific character geometric information. In this article, built upon a group of learnable geometric points, we propose a novel shape-driven attention alignment method that is able to obtain character-wise features. Concretely, we first design a corner detector to generate a shape map to guide the attention alignments explicitly, where a series of points can be learned to represent character-wise features flexibly. We then propose a dual-path network with a mutual learning and cooperating strategy that successfully combines CNN with a ViT-based model, leading to further accuracy improvement. We conduct extensive experiments to evaluate the proposed method on various scene text benchmarks, including six popular regular and irregular datasets, two more challenging datasets (i.e., WordArt and OST), and three Chinese datasets. Experimental results indicate that our method can achieve superior performance with a comparable model size against many state-of-the-art models.

Förderung

  • National Natural Science Foundation of China
  • Jiangsu Science and Technology Programme
  • European Union’s Horizon 2020 research and innovation programme
  • UK EPSRC under projects

Zitieren

APA

Hu, Y., Dong, B., Huang, K., Ding, L., Wang, W., Huang, X., & Wang, Q. (2024). Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(4), 1-20. https://doi.org/10.1145/3633517

Vancouver

Hu Y, Dong B, Huang K, Ding L, Wang W, Huang X, et al. Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(4):1-20. doi:10.1145/3633517

BibTeX
@article{Hu2024,
  author = {Hu, Yijie and Dong, Bin and Huang, Kaizhu and Ding, Lei and Wang, Wei and Huang, Xiaowei and Wang, Qiu-Feng},
  title = {Scene Text Recognition via Dual-path Network with Shape-driven Attention Alignment},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2024},
  volume = {20},
  number = {4},
  pages = {1--20},
  doi = {10.1145/3633517}
}
Kostenlos starten →