EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets

Mitra Malekzad, Fatih Kayaalp

Applied Sciences · 14.11.2026 · 16 (22) · 11303

DOI: 10.3390/app162211303 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 14.11.2026. Verfasst von Mitra Malekzad und 1 weiteren Autor:innen. Die Arbeit nennt 16 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The spread of fake news on digital platforms has increased the need for reliable automatic detection methods. This study presents a comparative empirical evaluation of five pretrained Transformer models—BERT, ALBERT, RoBERTa, XLM-RoBERTa, and ModernBERT—across three fake news detection datasets with different characteristics and class distributions: PolitiFact, GossipCop, and LIAR. To investigate class-imbalance handling, standard cross-entropy (CE), weighted cross-entropy (WCE), and random oversampling (ROS) were compared across the five architectures. Additional five-fold cross-validation experiments were conducted on PolitiFact to assess performance variability. The contribution of contextual metadata was examined on LIAR by comparing text-only and metadata-enhanced configurations. Performance was evaluated using accuracy, Macro-F1, Fake-class precision, recall, and F1-score, balanced accuracy, Matthews correlation coefficient, ROC-AUC, and average precision. In the original model comparison, RoBERTa achieved a Macro-F1 of 0.927 on PolitiFact and 0.792 on GossipCop. In the additional PolitiFact cross-validation experiments, RoBERTa with CE achieved the highest mean Macro-F1 of 0.888 ± 0.035. On LIAR, metadata improved Macro-F1 in 13 of the 15 matched model–strategy comparisons, with RoBERTa using WCE achieving the highest Macro-F1 of 0.655 among the controlled metadata-enhanced configurations. However, WCE and ROS did not consistently outperform CE, and improvements in minority-class recall were sometimes accompanied by reductions in other performance metrics. Overall, the findings demonstrate that the relative performance of Transformer architectures and imbalance-handling strategies varies across the evaluated datasets and configurations, highlighting the importance of controlled comparisons, complementary evaluation metrics, and cautious interpretation of cross-dataset differences.

Zitieren

APA

Malekzad, M., & Kayaalp, F. (2026). A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets. Applied Sciences, 16(22), 11303. https://doi.org/10.3390/app162211303

Vancouver

Malekzad M, Kayaalp F. A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets. Applied Sciences. 2026;16(22):11303. doi:10.3390/app162211303

BibTeX
@article{Malekzad2026,
  author = {Malekzad, Mitra and Kayaalp, Fatih},
  title = {A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {22},
  pages = {11303},
  doi = {10.3390/app162211303}
}
Kostenlos starten →