Multi-modal recommendation algorithm fusing visual and textual features
PLOS ONE · 29.06.2023 · 18 (6) · e0287927
DOI: 10.1371/journal.pone.0287927 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 29.06.2023. Verfasst von Xuefeng Hu und 3 weiteren Autor:innen. Beteiligte Einrichtung: The School of Electronics Engineering and Computer Science, Peking University, Beijing, China. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Science and Technology Foundation of Guizhou Province.
Abstract (Originalsprache)
In recommender systems, the lack of interaction data between users and items tends to lead to the problem of data sparsity and cold starts. Recently, the interest modeling frameworks incorporating multi-modal features are widely used in recommendation algorithms. These algorithms use image features and text features to extend the available information, which alleviate the data sparsity problem effectively, but they also have some limitations. On the one hand, multi-modal features of user interaction sequences are not considered in the interest modeling process. On the other hand, the aggregation of multi-modal features often employs simple aggregators, such as sums and concatenation, which do not distinguish the importance of different feature interactions. In this paper, to tackle this, we propose the FVTF (Fusing Visual and Textual Features) algorithm. First, we design a user history visual preference extraction module based on the Query-Key-Value attention to model users’ historical interests by using of visual features. Second, we design a feature fusion and interaction module based on the multi-head bit-wise attention to adaptively mine important feature combinations and update the higher-order attention fusion representation of features. We conduct experiments on the Movielens-1M dataset, and the experiments show that FVTF achieved the best performance compared with the benchmark recommendation algorithms.
Autor:innen und Institutionen
- The School of Electronics Engineering and Computer Science, Peking University, Beijing, China.
- The State Key Laboratory of Public Big Data, Guizhou University, Guiyang, China.
- The College of Computer Science and Technology, Guizhou University, Guiyang, China.
ORCID-Profile: Wenting Yu
Förderung
- National Natural Science Foundation of China
- Science and Technology Foundation of Guizhou Province
Zitieren
APA
Hu, X., Yu, W., Wu, Y., & Chen, Y. (2023). Multi-modal recommendation algorithm fusing visual and textual features. PLOS ONE, 18(6), e0287927. https://doi.org/10.1371/journal.pone.0287927
Vancouver
Hu X, Yu W, Wu Y, Chen Y. Multi-modal recommendation algorithm fusing visual and textual features. PLOS ONE. 2023;18(6):e0287927. doi:10.1371/journal.pone.0287927
BibTeX
@article{Hu2023,
author = {Hu, Xuefeng and Yu, Wenting and Wu, Yun and Chen, Yukang},
title = {Multi-modal recommendation algorithm fusing visual and textual features},
journal = {PLOS ONE},
year = {2023},
volume = {18},
number = {6},
pages = {e0287927},
doi = {10.1371/journal.pone.0287927}
}
EUVIMED