EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

AI‐Powered Scoring for Creative Thinking: Methods and Challenges in PISA Assessment

Ricardo Primi, Roger E. Beaty, Mathias Benedek, Denis Dumas, Peter Organisciak, John D. Patterson, Tiago Calico, Mario Piacentini

The Journal of Creative Behavior · 2026-03 · 60 (1)

DOI: 10.1002/jocb.70082 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Journal of Creative Behavior, online erschienen am 30.12.2025. Verfasst von Ricardo Primi und 7 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The introduction of the PISA 2022 Creative Thinking assessment underscores the growing need for scalable, valid, and reliable methods to evaluate creativity in international large‐scale assessments. Traditional human scoring, while nuanced, is time‐consuming, costly, and subject to inconsistencies. This paper explores recent advances in artificial intelligence (AI) and natural language processing (NLP)—particularly transformer‐based large language models (LLMs)—as promising alternatives for automated scoring. We review three methodological approaches: (1) unsupervised methods using semantic distance, (2) supervised fine‐tuning with labeled data, and (3) few−/zero‐shot learning using prompt‐based inference. Empirical findings from verbal and visual creative tasks show that AI‐based scoring systems can approximate human ratings with substantial accuracy ( r = 0.70–0.85), even across different languages and task formats. A case study using the PISA Book Covers task demonstrates convergence between AI and human scores, with reliability levels comparable to traditional scoring. However, key challenges remain, particularly regarding cross‐cultural comparability, bias mitigation, and interpretability. We discuss psychometric strategies (e.g., Many‐Facet Rasch Models) to model these issues and propose future directions, including scoring of distinct creativity dimensions and developing transparent, open‐source platforms. If rigorously validated, AI‐based scoring offers a feasible and equitable path forward for assessing creativity globally.

Förderung

  • Coordenação de Aperfeiçoamento de Pessoal de Nível Superior

Zitieren

APA

Primi, R., Beaty, R. E., Benedek, M., Dumas, D., Organisciak, P., Patterson, J. D., Calico, T., & Piacentini, M. (2026). AI‐Powered Scoring for Creative Thinking: Methods and Challenges in PISA Assessment. The Journal of Creative Behavior, 60(1). https://doi.org/10.1002/jocb.70082

Vancouver

Primi R, Beaty RE, Benedek M, Dumas D, Organisciak P, Patterson JD, et al. AI‐Powered Scoring for Creative Thinking: Methods and Challenges in PISA Assessment. The Journal of Creative Behavior. 2026;60(1). doi:10.1002/jocb.70082

BibTeX
@article{Primi2026,
  author = {Primi, Ricardo and Beaty, Roger E. and Benedek, Mathias and Dumas, Denis and Organisciak, Peter and Patterson, John D. and Calico, Tiago and Piacentini, Mario},
  title = {AI‐Powered Scoring for Creative Thinking: Methods and Challenges in PISA Assessment},
  journal = {The Journal of Creative Behavior},
  year = {2026},
  volume = {60},
  number = {1},
  doi = {10.1002/jocb.70082}
}
Kostenlos starten →