EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An image and text-based fake news detection with transfer learning

Esther Irawati Setiawan iD, Patrick Sutanto, Christian Nathaniel Purwanto, Joan Santoso, F X Ferdinandus, Nemuel Daniel Pah, Mauridhi Hery Purnomo

PLOS One · 2025

DOI: 10.1371/journal.pone.0324394 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Esther Irawati Setiawan und 6 weiteren Autor:innen. Beteiligte Einrichtung: Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia. Hauptthemen laut MEDLINE: Deception, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Fake news has emerged as a significant problem in today’s information age, threatening the reliability of information sources. Detecting fake news is crucial for maintaining trust and ensuring access to factual information. While deep learning offers solutions, most approaches focus on the text, neglecting the potential of visual information, which may contradict or misrepresent the accompanying text. This research proposes a multimodal classification approach that combines text and images to improve fake news detection, particularly in low-resource settings where labeled data is scarce. We leverage CLIP, a model that understands relationships between images and text, to extract features from both modalities. These features are concatenated and fed into a simple one-layer multi-layer perceptron (MLP) for classification. To enhance data efficiency, we apply LoRA (Low-Rank Adaptation), a parameter-efficient fine-tuning technique, to the CLIP model. We also explore the effects of integrating features from other models. The model achieves an 83% accuracy when using LoRA in classifying whether an image and its accompanying text constitute fake or factual news. These results highlight the potential of multimodal learning and efficient fine-tuning techniques for robust fake news detection, even with limited data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

DeceptionImage Processing, Computer-AssistedHumansDeep LearningAlgorithms

Autor:innen und Institutionen

  • Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia.
  • Electrical Engineering Department, University of Surabaya, Surabaya, Jawa Timur, Indonesia.
  • School of Engineering, RMIT University, Melbourne, Victoria, Australia.
  • Electrical Engineering Department, Institut Teknologi Sepuluh Nopember, Surabaya, Jawa Timur, Indonesia.

ORCID-Profile: Esther Irawati Setiawan

Zitieren

APA

Setiawan, E. I., Sutanto, P., Purwanto, C. N., Santoso, J., Ferdinandus, F. X., Pah, N. D., & Purnomo, M. H. (2025). An image and text-based fake news detection with transfer learning. PLOS One. https://doi.org/10.1371/journal.pone.0324394

Vancouver

Setiawan EI, Sutanto P, Purwanto CN, Santoso J, Ferdinandus FX, Pah ND, et al. An image and text-based fake news detection with transfer learning. PLOS One. 2025. doi:10.1371/journal.pone.0324394

BibTeX
@article{Setiawan2025,
  author = {Setiawan, Esther Irawati and Sutanto, Patrick and Purwanto, Christian Nathaniel and Santoso, Joan and Ferdinandus, F X and Pah, Nemuel Daniel and Purnomo, Mauridhi Hery},
  title = {An image and text-based fake news detection with transfer learning},
  journal = {PLOS One},
  year = {2025},
  doi = {10.1371/journal.pone.0324394}
}
Kostenlos starten →