EUVIMEDEUVIMED entdecken
Crossref

LiLTv2: Language-substitutable Layout-image Transformer for Visual Information Extraction

Jiapeng Wang, Zening Lin, Dayi Huang, Longfei Xiong, Lianwen Jin

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.03.2025 · 21 (3) · 1-27

DOI: 10.1145/3708351 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 19.02.2025. Verfasst von Jiapeng Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual Information Extraction (VIE) has experienced substantial growth and heightened interest due to its pivotal role in intelligent document processing. However, most existing related pre-trained models typically can only process the data from a certain (set of) language(s)—often just English, representing a distinct limitation. To solve it, we present a L anguage-subst i tutable L ayout-image T ransformer (LiLTv2). It can be pre-trained just once on mono-lingual documents and then collaborate with off-the-shelf textual models in other languages during fine-tuning. Firstly, LiLTv2 utilizes a new dual-stream model architecture, one stream for substitutable text information and the other for layout and image information. Then, LiLTv2 has improved upon the optimization strategy and the diverse tasks adopted in the pre-training stage. Finally, we innovatively propose a teacher-student knowledge distillation learning with segment-level multi-modal features named SegKD. Extensive experimental results on widely used benchmarks can demonstrate the superior effectiveness of our method.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Wang, J., Lin, Z., Huang, D., Xiong, L., & Jin, L. (2025). LiLTv2: Language-substitutable Layout-image Transformer for Visual Information Extraction. ACM Transactions on Multimedia Computing, Communications, and Applications, 21(3), 1-27. https://doi.org/10.1145/3708351

Vancouver

Wang J, Lin Z, Huang D, Xiong L, Jin L. LiLTv2: Language-substitutable Layout-image Transformer for Visual Information Extraction. ACM Transactions on Multimedia Computing, Communications, and Applications. 2025;21(3):1-27. doi:10.1145/3708351

BibTeX
@article{Wang2025,
  author = {Wang, Jiapeng and Lin, Zening and Huang, Dayi and Xiong, Longfei and Jin, Lianwen},
  title = {LiLTv2: Language-substitutable Layout-image Transformer for Visual Information Extraction},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2025},
  volume = {21},
  number = {3},
  pages = {1--27},
  doi = {10.1145/3708351}
}
Kostenlos starten →