An image and text-based fake news detection with transfer learning
PLOS One · 2025
DOI: 10.1371/journal.pone.0324394 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Esther Irawati Setiawan und 6 weiteren Autor:innen. Beteiligte Einrichtung: Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia. Hauptthemen laut MEDLINE: Deception, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Fake news has emerged as a significant problem in today’s information age, threatening the reliability of information sources. Detecting fake news is crucial for maintaining trust and ensuring access to factual information. While deep learning offers solutions, most approaches focus on the text, neglecting the potential of visual information, which may contradict or misrepresent the accompanying text. This research proposes a multimodal classification approach that combines text and images to improve fake news detection, particularly in low-resource settings where labeled data is scarce. We leverage CLIP, a model that understands relationships between images and text, to extract features from both modalities. These features are concatenated and fed into a simple one-layer multi-layer perceptron (MLP) for classification. To enhance data efficiency, we apply LoRA (Low-Rank Adaptation), a parameter-efficient fine-tuning technique, to the CLIP model. We also explore the effects of integrating features from other models. The model achieves an 83% accuracy when using LoRA in classifying whether an image and its accompanying text constitute fake or factual news. These results highlight the potential of multimodal learning and efficient fine-tuning techniques for robust fake news detection, even with limited data.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia.
- Electrical Engineering Department, University of Surabaya, Surabaya, Jawa Timur, Indonesia.
- School of Engineering, RMIT University, Melbourne, Victoria, Australia.
- Electrical Engineering Department, Institut Teknologi Sepuluh Nopember, Surabaya, Jawa Timur, Indonesia.
ORCID-Profile: Esther Irawati Setiawan
Zitieren
APA
Setiawan, E. I., Sutanto, P., Purwanto, C. N., Santoso, J., Ferdinandus, F. X., Pah, N. D., & Purnomo, M. H. (2025). An image and text-based fake news detection with transfer learning. PLOS One. https://doi.org/10.1371/journal.pone.0324394
Vancouver
Setiawan EI, Sutanto P, Purwanto CN, Santoso J, Ferdinandus FX, Pah ND, et al. An image and text-based fake news detection with transfer learning. PLOS One. 2025. doi:10.1371/journal.pone.0324394
BibTeX
@article{Setiawan2025,
author = {Setiawan, Esther Irawati and Sutanto, Patrick and Purwanto, Christian Nathaniel and Santoso, Joan and Ferdinandus, F X and Pah, Nemuel Daniel and Purnomo, Mauridhi Hery},
title = {An image and text-based fake news detection with transfer learning},
journal = {PLOS One},
year = {2025},
doi = {10.1371/journal.pone.0324394}
}
EUVIMED