EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Echoes in AI: Quantifying lack of plot diversity in LLM outputs

Weijia Xu iD, Nebojsa Jojic iD, Sudha Rao, Chris Brockett, Bill Dolan

Proceedings of the National Academy of Sciences · 02.09.2025 · 122 (35)

DOI: 10.1073/pnas.2504966122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, online erschienen am 28.08.2025. Verfasst von Weijia Xu und 4 weiteren Autor:innen. Beteiligte Einrichtung: Microsoft Research, Redmond, WA 98052. Hauptthemen laut MEDLINE: Language, Artificial Intelligence, Creativity. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With rapid advances in large language models (LLMs), there has been an increasing application of LLMs in creative content ideation and generation. A critical question emerges: can current LLMs provide ideas that are diverse enough to truly bolster collective creativity? We examine two state-of-the-art LLMs, GPT-4 and LLaMA-3, on story generation and discover that LLM-generated stories often consist of plot elements that are echoed across a number of generations. To quantify this phenomenon, we introduce the Sui Generis score, an automatic metric that measures the uniqueness of a plot element among alternative storylines generated using the same prompt under an LLM. Evaluating on 100 short stories, we find that LLM-generated stories often contain combinations of idiosyncratic plot elements echoed frequently across generations and across different LLMs, while plots from the original human-written stories are rarely recreated or even echoed in pieces. Moreover, our human evaluation shows that the ranking of Sui Generis scores among story segments correlates moderately with human judgment of surprise level, even though score computation is completely automatic without relying on human judgment.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LanguageArtificial IntelligenceCreativityHumans

Stichwörter der Autor:innen

AILLMcreativitydeep learningtext generation

Autor:innen und Institutionen

  • Microsoft Research, Redmond, WA 98052.

ORCID-Profile: Weijia Xu, Nebojsa Jojic

Zitieren

APA

Xu, W., Jojic, N., Rao, S., Brockett, C., & Dolan, B. (2025). Echoes in AI: Quantifying lack of plot diversity in LLM outputs. Proceedings of the National Academy of Sciences, 122(35). https://doi.org/10.1073/pnas.2504966122

Vancouver

Xu W, Jojic N, Rao S, Brockett C, Dolan B. Echoes in AI: Quantifying lack of plot diversity in LLM outputs. Proceedings of the National Academy of Sciences. 2025;122(35). doi:10.1073/pnas.2504966122

BibTeX
@article{Xu2025,
  author = {Xu, Weijia and Jojic, Nebojsa and Rao, Sudha and Brockett, Chris and Dolan, Bill},
  title = {Echoes in AI: Quantifying lack of plot diversity in LLM outputs},
  journal = {Proceedings of the National Academy of Sciences},
  year = {2025},
  volume = {122},
  number = {35},
  doi = {10.1073/pnas.2504966122}
}
Kostenlos starten →