Visual Entity Linking via Multi-modal Learning
Data Intelligence · 03.02.2022 · 4 (1) · 1-19
DOI: 10.1162/dint_a_00114 ↗Kurz gefasst
Fachartikel in der Zeitschrift Data Intelligence, online erschienen am 03.02.2022. Verfasst von Qiushuo Zheng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Existing visual scene understanding methods mainly focus on identifying coarse-grained concepts about the visual objects and their relationships, largely neglecting fine-grained scene understanding. In fact, many data-driven applications on the Web (e.g., news-reading and e-shopping) require accurate recognition of much less coarse concepts as entities and proper linking them to a knowledge graph (KG), which can take their performance to the next level. In light of this, in this paper, we identify a new research task: visual entity linking for fine-grained scene understanding. To accomplish the task, we first extract features of candidate entities from different modalities, i.e., visual features, textual features, and KG features. Then, we design a deep modal-attention neural network-based learning-to-rank method which aggregates all features and maps visual objects to the entities in KG. Extensive experimental results on the newly constructed dataset show that our proposed method is effective as it significantly improves the accuracy performance from 66.46% to 83.16% compared with baselines.
Zitieren
APA
Zheng, Q., Wen, H., Wang, M., & Qi, G. (2022). Visual Entity Linking via Multi-modal Learning. Data Intelligence, 4(1), 1-19. https://doi.org/10.1162/dint_a_00114
Vancouver
Zheng Q, Wen H, Wang M, Qi G. Visual Entity Linking via Multi-modal Learning. Data Intelligence. 2022;4(1):1-19. doi:10.1162/dint_a_00114
BibTeX
@article{Zheng2022,
author = {Zheng, Qiushuo and Wen, Hao and Wang, Meng and Qi, Guilin},
title = {Visual Entity Linking via Multi-modal Learning},
journal = {Data Intelligence},
year = {2022},
volume = {4},
number = {1},
pages = {1--19},
doi = {10.1162/dint_a_00114}
}
EUVIMED