EUVIMEDEUVIMED entdecken
Crossref

Multi-Grained Gradual Inference Model for Multimedia Event Extraction

Yang Liu, Fang Liu, Licheng Jiao, Qianyue Bao, Long Sun, Shuo Li, Lingling Li, Xu Liu

IEEE Transactions on Circuits and Systems for Video Technology · 2024-10 · 34 (10) · 10507-10520

DOI: 10.1109/tcsvt.2024.3402242 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-10. Verfasst von Yang Liu und 7 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 98 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Joint Fund Project of National Natural Science Foundation of China, China Postdoctoral Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the development of multimedia technology, events are usually presented in multimedia forms, thus multimedia event extraction (MEE) has become more and more important. Existing MEE works usually use simple strategies to align two modalities, making it difficult to precisely extract events and arguments in complex multimedia documents. To address this problem, we propose a novel Multi-grained Gradual Inference Model (MGIM) that focuses on inferring and interpreting events in complex multimedia structures in a coarse-to-fine manner. To efficiently integrate textual and visual modalities, we design a Coarse-grained Alignment (CA) module, which represents the two modalities in a graph structure and performs coarse-grained alignment. Based on the CA module, we further propose a Fine-grained Inference module (FI) that fine-grained aligns text and image by performing multiple rounds of gradual inference. MGIM provides a comprehensive interpretation of multimedia events at two information granularities (coarse and fine). Extensive experiments on the M2E2 dataset demonstrate the effectiveness of MGIM.

Förderung

  • National Natural Science Foundation of China
  • Joint Fund Project of National Natural Science Foundation of China
  • China Postdoctoral Science Foundation
  • Program for Cheung Kong Scholars and Innovative Research Team in University
  • Fund for Foreign Scholars in University Research and Teaching Programs
  • Key Scientific Technological Innovation Research Project by the Ministry of Education

Zitieren

APA

Liu, Y., Liu, F., Jiao, L., Bao, Q., Sun, L., Li, S., Li, L., & Liu, X. (2024). Multi-Grained Gradual Inference Model for Multimedia Event Extraction. IEEE Transactions on Circuits and Systems for Video Technology, 34(10), 10507-10520. https://doi.org/10.1109/tcsvt.2024.3402242

Vancouver

Liu Y, Liu F, Jiao L, Bao Q, Sun L, Li S, et al. Multi-Grained Gradual Inference Model for Multimedia Event Extraction. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(10):10507-10520. doi:10.1109/tcsvt.2024.3402242

BibTeX
@article{Liu2024,
  author = {Liu, Yang and Liu, Fang and Jiao, Licheng and Bao, Qianyue and Sun, Long and Li, Shuo and Li, Lingling and Liu, Xu},
  title = {Multi-Grained Gradual Inference Model for Multimedia Event Extraction},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2024},
  volume = {34},
  number = {10},
  pages = {10507--10520},
  doi = {10.1109/tcsvt.2024.3402242}
}
Kostenlos starten →