EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language

Şahin Korkmaz, Mustafa Alkan, Aynur Koçak

Electronics · 04.11.2026 · 15 (21) · 5108

DOI: 10.3390/electronics15215108 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 04.11.2026. Verfasst von Şahin Korkmaz und 2 weiteren Autor:innen. Die Arbeit nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Detectors of text generated by large language models can achieve high scores by exploiting shared source documents, data sources, or generator-specific artifacts rather than genuine linguistic signals. This is a serious concern for textual deepfakes written to mislead readers or impersonate human authors. We present a methodological audit of a Turkish human-versus-synthetic text classifier. The archived BERTurk model reproduced its near-perfect row-random test result (F1 = 0.9990), but 86.38% of test samples shared a root document with the training data. We then built a corrected 215,074-sample cohort from a single generator (Qwen2.5-7B-Instruct) and four development sources, using a group-disjoint split and three seeds. Within this cohort, BERTurk achieved F1 = 0.9958 ± 0.0001 (XLM-R: 0.9923 ± 0.0006), whereas cross-source testing lowered F1 to 0.8990. In a separate, prospectively frozen extension, nine new BERTurk models evaluated on held-out generator families reached a mean macro-F1 of 0.9379. However, on 500 independent Turkish Wikipedia texts, the same nine models showed a mean false-positive rate of 0.7384 and a balanced macro-F1 of 0.5500, failing all external-validity criteria. High in-cohort scores therefore do not indicate general performance beyond the evaluated generators and sources. The main contribution is a traceable evaluation protocol that separates historical, corrected, and failed-validity results.

Zitieren

APA

Korkmaz, Ş., Alkan, M., & Koçak, A. (2026). Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language. Electronics, 15(21), 5108. https://doi.org/10.3390/electronics15215108

Vancouver

Korkmaz Ş, Alkan M, Koçak A. Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language. Electronics. 2026;15(21):5108. doi:10.3390/electronics15215108

BibTeX
@article{Korkmaz2026,
  author = {Korkmaz, Şahin and Alkan, Mustafa and Koçak, Aynur},
  title = {Detection of Deepfake Texts Generated by Large Language Models: An Empirical Audit and Detection Framework for the Turkish Language},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {5108},
  doi = {10.3390/electronics15215108}
}
Kostenlos starten →