EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Evaluating Creative Output With Generative Artificial Intelligence: Comparing GPT Models and Human Experts in Idea Evaluation

Theresa Kranzle, Katelyn Sharratt

Creativity and Innovation Management · 2025-12 · 34 (4) · 991-1012

DOI: 10.1111/caim.70007 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Creativity and Innovation Management, online erschienen am 11.08.2025. Verfasst von Theresa Kranzle und 1 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 81 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Traditional techniques for evaluating creative outcomes are typically based on evaluations made by human experts. These methods suffer from challenges such as subjectivity, biases, limited availability, ‘crowding’, and high transaction costs. We propose that large language models (LLMs) can be used to overcome these shortcomings. However, there is a dearth of research comparing the performance of LLMs to traditional expert evaluations for evaluating creative outcomes such as ideas. Our study compares the alignment of expert evaluations with evaluations from the LLM GPT‐4. Our results reveal that to achieve moderate evaluation alignment with experts, LLMs require using a base framework and a spectrum‐based few‐shot prompt. We offer six theoretical contributions, shifting the focus from whether LLMs can evaluate to how specific design choices shape their alignment with human judgement. These insights are situated within broader frameworks from cognitive science, creativity theory, and machine learning. Furthermore, we outline six propositions for organizations interested in LLM‐supported evaluation methods. Key recommendations include utilizing base frameworks for large‐scale idea screening, establishing a database of evaluated ideas to optimize few‐shot performance, and leveraging AI–human collaboration for internal and external idea sourcing. Additionally, we highlight the need for privacy considerations when using third‐party LLMs for proprietary idea evaluations. This research contributes to innovation management literature by exploring methods for integrating LLM into creative evaluation processes to enhance scalability and efficiency while retaining evaluation quality.

Zitieren

APA

Kranzle, T., & Sharratt, K. (2025). Evaluating Creative Output With Generative Artificial Intelligence: Comparing GPT Models and Human Experts in Idea Evaluation. Creativity and Innovation Management, 34(4), 991-1012. https://doi.org/10.1111/caim.70007

Vancouver

Kranzle T, Sharratt K. Evaluating Creative Output With Generative Artificial Intelligence: Comparing GPT Models and Human Experts in Idea Evaluation. Creativity and Innovation Management. 2025;34(4):991-1012. doi:10.1111/caim.70007

BibTeX
@article{Kranzle2025,
  author = {Kranzle, Theresa and Sharratt, Katelyn},
  title = {Evaluating Creative Output With Generative Artificial Intelligence: Comparing GPT Models and Human Experts in Idea Evaluation},
  journal = {Creativity and Innovation Management},
  year = {2025},
  volume = {34},
  number = {4},
  pages = {991--1012},
  doi = {10.1111/caim.70007}
}
Kostenlos starten →