HKA: A Hierarchical Knowledge Alignment Framework for Multimodal Knowledge Graph Completion
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2024 · 20 (8) · 1-19
DOI: 10.1145/3664288 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 29.06.2024. Verfasst von Yunhui Xu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National High Level Hospital Clinical Research Funding, Beijing Natural Science Foundation.
Abstract (Originalsprache)
Recent years have witnessed the successful application of knowledge graph techniques in structured data processing, while how to incorporate knowledge from visual and textual modalities into knowledge graphs has been given less attention. To better organize them, Multimodal Knowledge Graphs (MKGs), comprising the structural triplets of traditional Knowledge Graphs (KGs) together with entity-related multimodal data (e.g., images and texts), have been introduced consecutively. However, it is still a great challenge to explore MKGs due to their inherent incompleteness. Although most existing Multimodal Knowledge Graph Completion (MKGC) approaches can infer missing triplets based on available factual triplets and multimodal information, they almost ignore the modal conflicts and supervisory effect, failing to achieve a more comprehensive understanding of entities. To address these issues, we propose a novel H ierarchical K nowledge A lignment ( HKA ) framework for MKGC. Specifically, a macro-knowledge alignment module is proposed to capture global semantic relevance between modalities for dealing with modal conflicts in MKG. Furthermore, a micro-knowledge alignment module is also developed to reveal the local consistency information through inter- and intra-modality supervisory effects more effectively. By integrating different modal predictions, a final decision can be made. Experimental results on three benchmark MKGC tasks have demonstrated the effectiveness of the proposed HKA framework.
Förderung
- National Key Research and Development Program of China
- National High Level Hospital Clinical Research Funding
- Beijing Natural Science Foundation
Zitieren
APA
Xu, Y., Li, Y., Xu, M., Zhu, Z., & Zhao, Y. (2024). HKA: A Hierarchical Knowledge Alignment Framework for Multimodal Knowledge Graph Completion. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(8), 1-19. https://doi.org/10.1145/3664288
Vancouver
Xu Y, Li Y, Xu M, Zhu Z, Zhao Y. HKA: A Hierarchical Knowledge Alignment Framework for Multimodal Knowledge Graph Completion. ACM Transactions on Multimedia Computing, Communications, and Applications. 2024;20(8):1-19. doi:10.1145/3664288
BibTeX
@article{Xu2024,
author = {Xu, Yunhui and Li, Youru and Xu, Muhao and Zhu, Zhenfeng and Zhao, Yao},
title = {HKA: A Hierarchical Knowledge Alignment Framework for Multimodal Knowledge Graph Completion},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2024},
volume = {20},
number = {8},
pages = {1--19},
doi = {10.1145/3664288}
}
EUVIMED