EUVIMEDEUVIMED entdecken
Crossref

Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education

Zafer Susoy

Pamukkale University Journal of Education · 21.08.2026 · (67) · 342-358

DOI: 10.9779/pauefd.1847553 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Pamukkale University Journal of Education, online erschienen am 21.08.2026. Verfasst von Zafer Susoy. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 32 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The proliferation of AI text generation tools in educational settings has created unprecedented challenges for academic integrity. As AI-generated content grows more sophisticated, traditional assessment methods may no longer suffice to distinguish human from machine-authored work. This mixed-methods study compared human reviewers, an AI scoring model, and plagiarism detection tools in identifying AI-generated essays. Sixteen essays were analyzed: eight by ELT master's students and eight AI-generated (ChatGPT 4o, Gemini 1.5, Llama), with four subsequently humanized. ELT was selected because writing proficiency and authentic voice are central disciplinary competencies. Two faculty reviewers, iThenticate, Turnitin, and a GPT-4o model evaluated all essays using an expert-reviewed analytic rubric covering coherence, originality, depth, writing style, and topic adherence. The AI model showed superior internal consistency (α = 0.923) versus human reviewers (α = 0.832). Turnitin achieved the highest accuracy (87.5%). Human reviewers performed inconsistently (62.5%; Fisher's Exact Test, p = .096). Inter-rater agreement was assessed via quadratic weighted kappa (QWK), as small sample sizes precluded a valid chi-square analysis. Digital tools currently outperform human judgment in detecting AI-generated content, yet limitations persist across all methods. These findings underscore the need for institutional AI policies, educator training, and hybrid human-technological assessment approaches.

Zitieren

APA

Susoy, Z. (2026). Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education. Pamukkale University Journal of Education, 342-358. https://doi.org/10.9779/pauefd.1847553

Vancouver

Susoy Z. Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education. Pamukkale University Journal of Education. 2026:342-358. doi:10.9779/pauefd.1847553

BibTeX
@article{Susoy2026,
  author = {Susoy, Zafer},
  title = {Evaluating Human Reviewers versus Artificial Intelligence: A Comparative Analysis of AI-Generated Essay Detection in Higher Education},
  journal = {Pamukkale University Journal of Education},
  year = {2026},
  number = {67},
  pages = {342--358},
  doi = {10.9779/pauefd.1847553}
}
Kostenlos starten →