EUVIMEDEUVIMED entdecken
Crossref

Who Grades More Consistently? Exploring AI vs. Human Teachers in the Assessment of Reflective Writing

Cecilia Ka Yuk Chan, Katherine K. W. Lee

AI-Enhanced Learning · 2025 · 1 (1) · 203-228

DOI: 10.70725/298318rdotfx ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI-Enhanced Learning, 2025. Verfasst von Cecilia Ka Yuk Chan und 1 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The assessment of reflective writing remains challenging due to the subjective nature of the task. This exploratory study compares the assessment of students’ reflective writing by six human higher education teachers and an AI tool, ChatGPT-4o. Using rubrics, 13 reflections from students participating in a three-week internship programme were assessed across four key categories: knowledge and application, critical thinking, personal development, and clarity. Fleiss’ Kappa, correlation analysis, and independent t-tests were conducted to evaluate inter-rater reliability, grading consistency, and alignment between human and AI grading, while qualitative analysis was used to understand any grading discrepancies. Findings reveal that the introduction of AI as an additional grader did not significantly alter overall grading consistency, though it exhibited more structured grading patterns with lower variability compared to human assessors. Qualitative analyses suggest that AI prioritises coherence and linguistic clarity, potentially overlooking deeper critical reflections. This study highlights the complexities of evaluating reflective writing and the potential of AI-assisted grading tools to complement human-made assessments. Findings offer insights into the evolving role of AI in assessment and the need for a balanced, hybrid grading approach that integrates both human judgment and AI’s efficiency. Keywords Artificial intelligence, reflective writing, assessment, AI-assisted grading, assessment of reflection, reflection literacy

Zitieren

APA

Chan, C. K. Y., & Lee, K. K. W. (2025). Who Grades More Consistently? Exploring AI vs. Human Teachers in the Assessment of Reflective Writing. AI-Enhanced Learning, 1(1), 203-228. https://doi.org/10.70725/298318rdotfx

Vancouver

Chan CKY, Lee KKW. Who Grades More Consistently? Exploring AI vs. Human Teachers in the Assessment of Reflective Writing. AI-Enhanced Learning. 2025;1(1):203-228. doi:10.70725/298318rdotfx

BibTeX
@article{Chan2025,
  author = {Chan, Cecilia Ka Yuk and Lee, Katherine K. W.},
  title = {Who Grades More Consistently? Exploring AI vs. Human Teachers in the Assessment of Reflective Writing},
  journal = {AI-Enhanced Learning},
  year = {2025},
  volume = {1},
  number = {1},
  pages = {203--228},
  doi = {10.70725/298318rdotfx}
}
Kostenlos starten →