Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education
Pamukkale University Journal of Education · 21.08.2026 · (67) · 342-358
DOI: 10.9779/pauefd.1847553 ↗Kurz gefasst
Fachartikel in der Zeitschrift Pamukkale University Journal of Education, online erschienen am 21.08.2026. Verfasst von Zafer Susoy. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 32 Literaturangaben.
Abstract (Originalsprache)
The proliferation of AI text generation tools in educational settings has created unprecedented challenges for academic integrity. As AI-generated content grows more sophisticated, traditional assessment methods may no longer suffice to distinguish human from machine-authored work. This mixed-methods study compared human reviewers, an AI scoring model, and plagiarism detection tools in identifying AI-generated essays. Sixteen essays were analyzed: eight by ELT master's students and eight AI-generated (ChatGPT 4o, Gemini 1.5, Llama), with four subsequently humanized. ELT was selected because writing proficiency and authentic voice are central disciplinary competencies. Two faculty reviewers, iThenticate, Turnitin, and a GPT-4o model evaluated all essays using an expert-reviewed analytic rubric covering coherence, originality, depth, writing style, and topic adherence. The AI model showed superior internal consistency (α = 0.923) versus human reviewers (α = 0.832). Turnitin achieved the highest accuracy (87.5%). Human reviewers performed inconsistently (62.5%; Fisher's Exact Test, p = .096). Inter-rater agreement was assessed via quadratic weighted kappa (QWK), as small sample sizes precluded a valid chi-square analysis. Digital tools currently outperform human judgment in detecting AI-generated content, yet limitations persist across all methods. These findings underscore the need for institutional AI policies, educator training, and hybrid human-technological assessment approaches.
Zitieren
APA
Susoy, Z. (2026). Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education. Pamukkale University Journal of Education, 342-358. https://doi.org/10.9779/pauefd.1847553
Vancouver
Susoy Z. Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education. Pamukkale University Journal of Education. 2026:342-358. doi:10.9779/pauefd.1847553
BibTeX
@article{Susoy2026,
author = {Susoy, Zafer},
title = {Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education},
journal = {Pamukkale University Journal of Education},
year = {2026},
number = {67},
pages = {342--358},
doi = {10.9779/pauefd.1847553}
}
EUVIMED