EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multi-modal recommendation algorithm fusing visual and textual features

Xuefeng Hu, Wenting Yu iD, Yun Wu, Yukang Chen

PLOS ONE · 29.06.2023 · 18 (6) · e0287927

DOI: 10.1371/journal.pone.0287927 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 29.06.2023. Verfasst von Xuefeng Hu und 3 weiteren Autor:innen. Beteiligte Einrichtung: The School of Electronics Engineering and Computer Science, Peking University, Beijing, China. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Science and Technology Foundation of Guizhou Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recommender systems, the lack of interaction data between users and items tends to lead to the problem of data sparsity and cold starts. Recently, the interest modeling frameworks incorporating multi-modal features are widely used in recommendation algorithms. These algorithms use image features and text features to extend the available information, which alleviate the data sparsity problem effectively, but they also have some limitations. On the one hand, multi-modal features of user interaction sequences are not considered in the interest modeling process. On the other hand, the aggregation of multi-modal features often employs simple aggregators, such as sums and concatenation, which do not distinguish the importance of different feature interactions. In this paper, to tackle this, we propose the FVTF (Fusing Visual and Textual Features) algorithm. First, we design a user history visual preference extraction module based on the Query-Key-Value attention to model users’ historical interests by using of visual features. Second, we design a feature fusion and interaction module based on the multi-head bit-wise attention to adaptively mine important feature combinations and update the higher-order attention fusion representation of features. We conduct experiments on the Movielens-1M dataset, and the experiments show that FVTF achieved the best performance compared with the benchmark recommendation algorithms.

Autor:innen und Institutionen

  • The School of Electronics Engineering and Computer Science, Peking University, Beijing, China.
  • The State Key Laboratory of Public Big Data, Guizhou University, Guiyang, China.
  • The College of Computer Science and Technology, Guizhou University, Guiyang, China.

ORCID-Profile: Wenting Yu

Förderung

  • National Natural Science Foundation of China
  • Science and Technology Foundation of Guizhou Province

Zitieren

APA

Hu, X., Yu, W., Wu, Y., & Chen, Y. (2023). Multi-modal recommendation algorithm fusing visual and textual features. PLOS ONE, 18(6), e0287927. https://doi.org/10.1371/journal.pone.0287927

Vancouver

Hu X, Yu W, Wu Y, Chen Y. Multi-modal recommendation algorithm fusing visual and textual features. PLOS ONE. 2023;18(6):e0287927. doi:10.1371/journal.pone.0287927

BibTeX
@article{Hu2023,
  author = {Hu, Xuefeng and Yu, Wenting and Wu, Yun and Chen, Yukang},
  title = {Multi-modal recommendation algorithm fusing visual and textual features},
  journal = {PLOS ONE},
  year = {2023},
  volume = {18},
  number = {6},
  pages = {e0287927},
  doi = {10.1371/journal.pone.0287927}
}
Kostenlos starten →