EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Improving AI-assisted grading reliability through explicit rubric and prompt engineering

Deepshikha Deepshikha, Navpreet Singh

Frontiers in Education · 18.11.2026 · 11

DOI: 10.3389/feduc.2026.1968847 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Education, online erschienen am 18.11.2026. Verfasst von Deepshikha Deepshikha und 1 weiteren Autor:innen. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The use of artificial intelligence (AI) to support assessment and feedback in higher education has attracted growing interest, but concerns remain about the consistency and appropriateness of AI-generated grades in summative contexts. This study examined whether a combined calibration strategy involving a more explicit rubric and marking prompt could improve agreement between AI-assisted grading and educator judgement in a laboratory-based STEM assessment. We compared educator-assigned marks for 40 graduate Electrochemical Energy Storage Systems laboratory reports with marks generated by GPT-5 under pre- and post-calibration configurations, and examined agreement using complementary statistical and visual analyses. Following calibration, agreement with educator marks improved across measures of score correspondence and absolute agreement: the intraclass correlation coefficient increased from 0.818 to 0.928, the mean AI-educator difference decreased from +1.625 to +1.041 marks, and no post-calibration report differed from the educator mark by more than 10 marks. These findings suggest that the way assessment criteria, evidence requirements, and marking rules are specified can materially influence AI-human grading agreement. However, the revised rubric and prompt were introduced together, calibration drew on discrepancies observed in the same 40 reports used for post-calibration evaluation, and one educator's marks served as the reference. The results therefore support improved in-sample agreement under a human-supervised calibration strategy, rather than establishing grading validity or interchangeability for autonomous summative decisions.

Zitieren

APA

Deepshikha, D., & Singh, N. (2026). Improving AI-assisted grading reliability through explicit rubric and prompt engineering. Frontiers in Education, 11. https://doi.org/10.3389/feduc.2026.1968847

Vancouver

Deepshikha D, Singh N. Improving AI-assisted grading reliability through explicit rubric and prompt engineering. Frontiers in Education. 2026;11. doi:10.3389/feduc.2026.1968847

BibTeX
@article{Deepshikha2026,
  author = {Deepshikha, Deepshikha and Singh, Navpreet},
  title = {Improving AI-assisted grading reliability through explicit rubric and prompt engineering},
  journal = {Frontiers in Education},
  year = {2026},
  volume = {11},
  doi = {10.3389/feduc.2026.1968847}
}
Kostenlos starten →