EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating Creativity: Can LLMs Be Good Evaluators in Creative Writing Tasks?

Sungeun Kim, Dongsuk Oh

Applied Sciences · 10.03.2025 · 15 (6) · 2971

DOI: 10.3390/app15062971 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 10.03.2025. Verfasst von Sungeun Kim und 1 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Kyungpook National University Research Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The evaluation of creative writing has long been a complex and subjective process, made even more intriguing by the rise of advanced Artificial Intelligence (AI) tools like Large Language Models (LLMs). This study evaluates the potential of LLMs as reliable and consistent evaluators of creative texts, directly comparing their performance with traditional human evaluations. The analysis focuses on key creative criteria, including fluency, flexibility, elaboration, originality, usefulness, and specific creativity strategies. Results demonstrate that LLMs provide consistent and objective evaluations, achieving higher Inter-Annotator Agreement (IAA) compared with human evaluators. However, LLMs face limitations in recognizing nuanced, culturally specific, and context-dependent aspects of creativity. Conversely, human evaluators, despite lower consistency and higher subjectivity, exhibit strengths in capturing deeper contextual insights. These findings highlight the need for the further refinement of LLMs to address the complexities of creative writing evaluation.

Förderung

  • Kyungpook National University Research Fund

Zitieren

APA

Kim, S., & Oh, D. (2025). Evaluating Creativity: Can LLMs Be Good Evaluators in Creative Writing Tasks?. Applied Sciences, 15(6), 2971. https://doi.org/10.3390/app15062971

Vancouver

Kim S, Oh D. Evaluating Creativity: Can LLMs Be Good Evaluators in Creative Writing Tasks?. Applied Sciences. 2025;15(6):2971. doi:10.3390/app15062971

BibTeX
@article{Kim2025,
  author = {Kim, Sungeun and Oh, Dongsuk},
  title = {Evaluating Creativity: Can LLMs Be Good Evaluators in Creative Writing Tasks?},
  journal = {Applied Sciences},
  year = {2025},
  volume = {15},
  number = {6},
  pages = {2971},
  doi = {10.3390/app15062971}
}
Kostenlos starten →