A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets
Applied Sciences · 14.11.2026 · 16 (22) · 11303
DOI: 10.3390/app162211303 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 14.11.2026. Verfasst von Mitra Malekzad und 1 weiteren Autor:innen. Die Arbeit nennt 16 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The spread of fake news on digital platforms has increased the need for reliable automatic detection methods. This study presents a comparative empirical evaluation of five pretrained Transformer models—BERT, ALBERT, RoBERTa, XLM-RoBERTa, and ModernBERT—across three fake news detection datasets with different characteristics and class distributions: PolitiFact, GossipCop, and LIAR. To investigate class-imbalance handling, standard cross-entropy (CE), weighted cross-entropy (WCE), and random oversampling (ROS) were compared across the five architectures. Additional five-fold cross-validation experiments were conducted on PolitiFact to assess performance variability. The contribution of contextual metadata was examined on LIAR by comparing text-only and metadata-enhanced configurations. Performance was evaluated using accuracy, Macro-F1, Fake-class precision, recall, and F1-score, balanced accuracy, Matthews correlation coefficient, ROC-AUC, and average precision. In the original model comparison, RoBERTa achieved a Macro-F1 of 0.927 on PolitiFact and 0.792 on GossipCop. In the additional PolitiFact cross-validation experiments, RoBERTa with CE achieved the highest mean Macro-F1 of 0.888 ± 0.035. On LIAR, metadata improved Macro-F1 in 13 of the 15 matched model–strategy comparisons, with RoBERTa using WCE achieving the highest Macro-F1 of 0.655 among the controlled metadata-enhanced configurations. However, WCE and ROS did not consistently outperform CE, and improvements in minority-class recall were sometimes accompanied by reductions in other performance metrics. Overall, the findings demonstrate that the relative performance of Transformer architectures and imbalance-handling strategies varies across the evaluated datasets and configurations, highlighting the importance of controlled comparisons, complementary evaluation metrics, and cautious interpretation of cross-dataset differences.
Zitieren
APA
Malekzad, M., & Kayaalp, F. (2026). A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets. Applied Sciences, 16(22), 11303. https://doi.org/10.3390/app162211303
Vancouver
Malekzad M, Kayaalp F. A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets. Applied Sciences. 2026;16(22):11303. doi:10.3390/app162211303
BibTeX
@article{Malekzad2026,
author = {Malekzad, Mitra and Kayaalp, Fatih},
title = {A Comparative Study of Transformer Models for Fake News Detection on Imbalanced Datasets},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {22},
pages = {11303},
doi = {10.3390/app162211303}
}
EUVIMED