EUVIMEDEUVIMED entdecken
Crossref

Reimagining Leadership Assessment: A Comparative Study of Human and AI Scoring Using ChatGPT in School Principal Training

Mingchuan Hsieh

Journal of Advanced Computational Intelligence and Intelligent Informatics · 20.03.2026 · 30 (2) · 348-353

DOI: 10.20965/jaciii.2026.p0348 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Advanced Computational Intelligence and Intelligent Informatics, online erschienen am 20.03.2026. Verfasst von Mingchuan Hsieh. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 14 Literaturangaben. Gefördert von: Ministry of Science and Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this study, the application of ChatGPT in the evaluation of pre-service principals within Taiwan’s national school leadership training program was examined. As generative AI technologies become increasingly embedded in education, understanding their role in professional assessment is essential. This research explored whether AI-generated scores align with human ratings, and whether ChatGPT can serve as a reliable tool for providing formative feedback. A total of 131 pre-service principals submitted School Improvement Plans, which were scored by both human evaluators (mentor principals and external scholars) and ChatGPT. The scoring rubric included seven dimensions of leadership competency, with both parties rating assignments on a five-point scale. Descriptive statistics, Pearson correlations, and intraclass correlation coefficients (ICCs) were used to compare scoring patterns, consistency, and reliability. Findings show that human raters consistently assigned higher and more variable scores than ChatGPT, which produced more conservative and evenly distributed ratings. Although both AI and human scores showed internal construct coherence, correlation patterns suggested that ChatGPT applied a more differentiated, data-driven evaluation logic. ICC analysis revealed moderate single-rater reliability and high average-rater reliability for both human and AI assessments. The results indicate that ChatGPT holds potential as a supplementary assessment assistant, particularly in delivering structured, rubric-based feedback. However, discrepancies in scoring tendencies and contextual interpretation underscore the need for human oversight. The findings of this study offer implications for the thoughtful integration of AI in leadership development and broader educational assessment systems.

Förderung

  • Ministry of Science and Technology

Zitieren

APA

Hsieh, M. (2026). Reimagining Leadership Assessment: A Comparative Study of Human and AI Scoring Using ChatGPT in School Principal Training. Journal of Advanced Computational Intelligence and Intelligent Informatics, 30(2), 348-353. https://doi.org/10.20965/jaciii.2026.p0348

Vancouver

Hsieh M. Reimagining Leadership Assessment: A Comparative Study of Human and AI Scoring Using ChatGPT in School Principal Training. Journal of Advanced Computational Intelligence and Intelligent Informatics. 2026;30(2):348-353. doi:10.20965/jaciii.2026.p0348

BibTeX
@article{Hsieh2026,
  author = {Hsieh, Mingchuan},
  title = {Reimagining Leadership Assessment: A Comparative Study of Human and AI Scoring Using ChatGPT in School Principal Training},
  journal = {Journal of Advanced Computational Intelligence and Intelligent Informatics},
  year = {2026},
  volume = {30},
  number = {2},
  pages = {348--353},
  doi = {10.20965/jaciii.2026.p0348}
}
Kostenlos starten →