EUVIMEDEUVIMED entdecken
Crossref

Dynamic Multimodal Fusion via Meta-Learning Towards Micro-Video Recommendation

Han Liu, Yinwei Wei, Fan Liu, Wenjie Wang, Liqiang Nie, Tat-Seng Chua

ACM Transactions on Information Systems · 31.03.2024 · 42 (2) · 1-26

DOI: 10.1145/3617827 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Information Systems, online erschienen am 08.11.2023. Verfasst von Han Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Program of China Scholarship Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal information (e.g., visual, acoustic, and textual) has been widely used to enhance representation learning for micro-video recommendation. For integrating multimodal information into a joint representation of micro-video, multimodal fusion plays a vital role in the existing micro-video recommendation approaches. However, the static multimodal fusion used in previous studies is insufficient to model the various relationships among multimodal information of different micro-videos. In this article, we develop a novel meta-learning-based multimodal fusion framework called Meta Multimodal Fusion (MetaMMF), which dynamically assigns parameters to the multimodal fusion function for each micro-video during its representation learning. Specifically, MetaMMF regards the multimodal fusion of each micro-video as an independent task. Based on the meta information extracted from the multimodal features of the input task, MetaMMF parameterizes a neural network as the item-specific fusion function via a meta learner. We perform extensive experiments on three benchmark datasets, demonstrating the significant improvements over several state-of-the-art multimodal recommendation models, like MMGCN, LATTICE, and InvRL. Furthermore, we lighten our model by adopting canonical polyadic decomposition to improve the training efficiency, and validate its effectiveness through experimental results. Codes are available at https://github.com/hanliu95/MetaMMF .

Förderung

  • National Natural Science Foundation of China
  • Program of China Scholarship Council

Zitieren

APA

Liu, H., Wei, Y., Liu, F., Wang, W., Nie, L., & Chua, T. (2024). Dynamic Multimodal Fusion via Meta-Learning Towards Micro-Video Recommendation. ACM Transactions on Information Systems, 42(2), 1-26. https://doi.org/10.1145/3617827

Vancouver

Liu H, Wei Y, Liu F, Wang W, Nie L, Chua T. Dynamic Multimodal Fusion via Meta-Learning Towards Micro-Video Recommendation. ACM Transactions on Information Systems. 2024;42(2):1-26. doi:10.1145/3617827

BibTeX
@article{Liu2024,
  author = {Liu, Han and Wei, Yinwei and Liu, Fan and Wang, Wenjie and Nie, Liqiang and Chua, Tat-Seng},
  title = {Dynamic Multimodal Fusion via Meta-Learning Towards Micro-Video Recommendation},
  journal = {ACM Transactions on Information Systems},
  year = {2024},
  volume = {42},
  number = {2},
  pages = {1--26},
  doi = {10.1145/3617827}
}
Kostenlos starten →