EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

ChatGPT as a Stable and Fair Tool for Automated Essay Scoring

Francisco García-Varela, Miguel Nussbaum, Marcelo Mendoza, Carolina Martínez-Troncoso, Zvi Bekerman

Education Sciences · 2025-07-23 · 15 (8) · 946

DOI: 10.3390/educsci15080946 ↗

At a glance

Journal article in Education Sciences, published online on 2025-07-23. Written by Francisco García-Varela and 4 co-authors. Cited 28 times so far (Crossref); 48 references. The full text is freely available (open access, CC BY 4.0). Funded by: Agencia Nacional de Investigación y Desarrollo (ANID), Millennium Institute for Foundational Research on Data (IMFD), National Center for Artificial Intelligence (CENIA).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

The evaluation of open-ended questions is typically performed by human instructors using predefined criteria to uphold academic standards. However, manual grading presents challenges, including high costs, rater fatigue, and potential bias, prompting interest in automated essay scoring systems. While automated essay scoring tools can assess content, coherence, and grammar, discrepancies between human and automated scoring have raised concerns about their reliability as standalone evaluators. Large language models like ChatGPT offer new possibilities, but their consistency and fairness in feedback remain underexplored. This study investigates whether ChatGPT can provide stable and fair essay scoring—specifically, whether identical student responses receive consistent evaluations across multiple AI interactions using the same criteria. The study was conducted in two marketing courses at an engineering school in Chile, involving 40 students. Results showed that ChatGPT, when unprompted or using minimal guidance, produced volatile grades and shifting criteria. Incorporating the instructor’s rubric reduced this variability but did not eliminate it. Only after providing an example-rich rubric, a standardized output format, low temperature settings, and a normalization process based on decision tables did ChatGPT-4o demonstrate consistent and fair grading. Based on these findings, we developed a scalable algorithm that automatically generates effective grading rubrics and decision tables with minimal human input. The added value of this work lies in the development of a scalable algorithm capable of automatically generating normalized rubrics and decision tables for new questions, thereby extending the accessibility and reliability of automated assessment.

Funding

  • Agencia Nacional de Investigación y Desarrollo (ANID)
  • Millennium Institute for Foundational Research on Data (IMFD)
  • National Center for Artificial Intelligence (CENIA)

Cite

APA

García-Varela, F., Nussbaum, M., Mendoza, M., Martínez-Troncoso, C., & Bekerman, Z. (2025). ChatGPT as a Stable and Fair Tool for Automated Essay Scoring. Education Sciences, 15(8), 946. https://doi.org/10.3390/educsci15080946

Vancouver

García-Varela F, Nussbaum M, Mendoza M, Martínez-Troncoso C, Bekerman Z. ChatGPT as a Stable and Fair Tool for Automated Essay Scoring. Education Sciences. 2025;15(8):946. doi:10.3390/educsci15080946

BibTeX
@article{GarcaVarela2025,
  author = {García-Varela, Francisco and Nussbaum, Miguel and Mendoza, Marcelo and Martínez-Troncoso, Carolina and Bekerman, Zvi},
  title = {ChatGPT as a Stable and Fair Tool for Automated Essay Scoring},
  journal = {Education Sciences},
  year = {2025},
  volume = {15},
  number = {8},
  pages = {946},
  doi = {10.3390/educsci15080946}
}
Start for free →