Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language
Electronics · 04.11.2026 · 15 (21) · 5108
DOI: 10.3390/electronics15215108 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 04.11.2026. Verfasst von Şahin Korkmaz und 2 weiteren Autor:innen. Die Arbeit nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Detectors of text generated by large language models can achieve high scores by exploiting shared source documents, data sources, or generator-specific artifacts rather than genuine linguistic signals. This is a serious concern for textual deepfakes written to mislead readers or impersonate human authors. We present a methodological audit of a Turkish human-versus-synthetic text classifier. The archived BERTurk model reproduced its near-perfect row-random test result (F1 = 0.9990), but 86.38% of test samples shared a root document with the training data. We then built a corrected 215,074-sample cohort from a single generator (Qwen2.5-7B-Instruct) and four development sources, using a group-disjoint split and three seeds. Within this cohort, BERTurk achieved F1 = 0.9958 ± 0.0001 (XLM-R: 0.9923 ± 0.0006), whereas cross-source testing lowered F1 to 0.8990. In a separate, prospectively frozen extension, nine new BERTurk models evaluated on held-out generator families reached a mean macro-F1 of 0.9379. However, on 500 independent Turkish Wikipedia texts, the same nine models showed a mean false-positive rate of 0.7384 and a balanced macro-F1 of 0.5500, failing all external-validity criteria. High in-cohort scores therefore do not indicate general performance beyond the evaluated generators and sources. The main contribution is a traceable evaluation protocol that separates historical, corrected, and failed-validity results.
Zitieren
APA
Korkmaz, Ş., Alkan, M., & Koçak, A. (2026). Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language. Electronics, 15(21), 5108. https://doi.org/10.3390/electronics15215108
Vancouver
Korkmaz Ş, Alkan M, Koçak A. Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language. Electronics. 2026;15(21):5108. doi:10.3390/electronics15215108
BibTeX
@article{Korkmaz2026,
author = {Korkmaz, Şahin and Alkan, Mustafa and Koçak, Aynur},
title = {Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language},
journal = {Electronics},
year = {2026},
volume = {15},
number = {21},
pages = {5108},
doi = {10.3390/electronics15215108}
}
EUVIMED