EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Visual Entity Linking via Multi-modal Learning

Qiushuo Zheng, Hao Wen, Meng Wang, Guilin Qi

Data Intelligence · 03.02.2022 · 4 (1) · 1-19

DOI: 10.1162/dint_a_00114 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Data Intelligence, online erschienen am 03.02.2022. Verfasst von Qiushuo Zheng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing visual scene understanding methods mainly focus on identifying coarse-grained concepts about the visual objects and their relationships, largely neglecting fine-grained scene understanding. In fact, many data-driven applications on the Web (e.g., news-reading and e-shopping) require accurate recognition of much less coarse concepts as entities and proper linking them to a knowledge graph (KG), which can take their performance to the next level. In light of this, in this paper, we identify a new research task: visual entity linking for fine-grained scene understanding. To accomplish the task, we first extract features of candidate entities from different modalities, i.e., visual features, textual features, and KG features. Then, we design a deep modal-attention neural network-based learning-to-rank method which aggregates all features and maps visual objects to the entities in KG. Extensive experimental results on the newly constructed dataset show that our proposed method is effective as it significantly improves the accuracy performance from 66.46% to 83.16% compared with baselines.

Zitieren

APA

Zheng, Q., Wen, H., Wang, M., & Qi, G. (2022). Visual Entity Linking via Multi-modal Learning. Data Intelligence, 4(1), 1-19. https://doi.org/10.1162/dint_a_00114

Vancouver

Zheng Q, Wen H, Wang M, Qi G. Visual Entity Linking via Multi-modal Learning. Data Intelligence. 2022;4(1):1-19. doi:10.1162/dint_a_00114

BibTeX
@article{Zheng2022,
  author = {Zheng, Qiushuo and Wen, Hao and Wang, Meng and Qi, Guilin},
  title = {Visual Entity Linking via Multi-modal Learning},
  journal = {Data Intelligence},
  year = {2022},
  volume = {4},
  number = {1},
  pages = {1--19},
  doi = {10.1162/dint_a_00114}
}
Kostenlos starten →