Multi-Grained Gradual Inference Model for Multimedia Event Extraction
IEEE Transactions on Circuits and Systems for Video Technology · 2024-10 · 34 (10) · 10507-10520
DOI: 10.1109/tcsvt.2024.3402242 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-10. Verfasst von Yang Liu und 7 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 98 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Joint Fund Project of National Natural Science Foundation of China, China Postdoctoral Science Foundation.
Abstract (Originalsprache)
With the development of multimedia technology, events are usually presented in multimedia forms, thus multimedia event extraction (MEE) has become more and more important. Existing MEE works usually use simple strategies to align two modalities, making it difficult to precisely extract events and arguments in complex multimedia documents. To address this problem, we propose a novel Multi-grained Gradual Inference Model (MGIM) that focuses on inferring and interpreting events in complex multimedia structures in a coarse-to-fine manner. To efficiently integrate textual and visual modalities, we design a Coarse-grained Alignment (CA) module, which represents the two modalities in a graph structure and performs coarse-grained alignment. Based on the CA module, we further propose a Fine-grained Inference module (FI) that fine-grained aligns text and image by performing multiple rounds of gradual inference. MGIM provides a comprehensive interpretation of multimedia events at two information granularities (coarse and fine). Extensive experiments on the M2E2 dataset demonstrate the effectiveness of MGIM.
Förderung
- National Natural Science Foundation of China
- Joint Fund Project of National Natural Science Foundation of China
- China Postdoctoral Science Foundation
- Program for Cheung Kong Scholars and Innovative Research Team in University
- Fund for Foreign Scholars in University Research and Teaching Programs
- Key Scientific Technological Innovation Research Project by the Ministry of Education
Zitieren
APA
Liu, Y., Liu, F., Jiao, L., Bao, Q., Sun, L., Li, S., Li, L., & Liu, X. (2024). Multi-Grained Gradual Inference Model for Multimedia Event Extraction. IEEE Transactions on Circuits and Systems for Video Technology, 34(10), 10507-10520. https://doi.org/10.1109/tcsvt.2024.3402242
Vancouver
Liu Y, Liu F, Jiao L, Bao Q, Sun L, Li S, et al. Multi-Grained Gradual Inference Model for Multimedia Event Extraction. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(10):10507-10520. doi:10.1109/tcsvt.2024.3402242
BibTeX
@article{Liu2024,
author = {Liu, Yang and Liu, Fang and Jiao, Licheng and Bao, Qianyue and Sun, Long and Li, Shuo and Li, Lingling and Liu, Xu},
title = {Multi-Grained Gradual Inference Model for Multimedia Event Extraction},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {10},
pages = {10507--10520},
doi = {10.1109/tcsvt.2024.3402242}
}
EUVIMED