EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Writing creativity, cohesion, and formal linguistic competence in LLMs: A comparative evaluation based on English and Chinese continuation writing

Yao Zhang, Tongquan Zhou iD, Yulu Li, Yizhong Xu, Taohui Li iD, Juan Wang iD

PLOS One · 22.06.2026 · 21 (6) · e0335185

DOI: 10.1371/journal.pone.0335185 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift PLOS One, online erschienen am 22.06.2026. Verfasst von Yao Zhang und 5 weiteren Autor:innen. Beteiligte Einrichtung: School of Foreign Languages, Southeast University, Nanjing, China. Hauptthemen laut MEDLINE: Creativity, Large Language Models, Linguistics, Writing. Die Arbeit nennt 76 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: SEU Innovation Capability Enhancement Plan for Doctoral Students, Jiangsu Province Degree and Postgraduate Education Teaching Reform Project, Postgraduate Research & Practice Innovation Program of Jiangsu-Province.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advancements in large language models (LLMs), such as GPT-3.5, GPT-4, and Chinese models like ERNIE and SparkDesk (SPARK), have shown remarkable potential in story generation. However, a direct comparison between ChatGPTs, ERNIE, and SPARK (pretrained with dominant Chinese Corpus, referred to herein as C-dominant LLMs) remains underexplored in this domain, particularly regarding narrative writing continuation ability. Identifying such difference would be beneficial to teachers to adopt LLMs in writing practice. This study addresses this issue by comparing ChatGPTs and C-dominant LLMs in story continuation tasks, with a focus on cohesion, creativity, and formal linguistic competence. We harnessed advanced analytical tools, including Coh-Metrix and CTAP for cohesion and LIWC-22 and CLIWC for creativity, to assess the performance of GPT-3.5, GPT-4, ERNIE, and SPARK across English and Chinese continuations. Results indicated that C-dominant LLMs surpassed ChatGPTs in referential and deep cohesion for English story generation, while ChatGPTs excelled in latent semantic analysis. Conversely, ChatGPTs demonstrated superior performance in cohesion across all metrics for Chinese story generation. In terms of creativity, GPT models behaved better in image for English texts and voice for both English and Chinese, whereas C-dominant LLMs excelled in image for Chinese continuations. For formal linguistic competence, GPTs showed better performance in English tasks, while C-dominant LLMs in Chinese tasks. These findings provide insights for optimizing LLMs in multilingual contexts and offer guidance for their application in writing and translation based on diverse languages.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

CreativityLarge Language ModelsLinguisticsWritingHumansChina

Autor:innen und Institutionen

  • School of Foreign Languages, Southeast University, Nanjing, China.
  • College of Chinese Literature and Language, Qufu Normal University, Qufu, China.
  • College of Foreign Languages, Nanjing University of Aeronautics and Astronautics, Nanjing, China.
  • College of Foreign Languages, Qufu Normal University, Qufu, China.

ORCID-Profile: Tongquan Zhou, Taohui Li, Juan Wang

Förderung

  • SEU Innovation Capability Enhancement Plan for Doctoral Students
  • Jiangsu Province Degree and Postgraduate Education Teaching Reform Project
  • Postgraduate Research & Practice Innovation Program of Jiangsu-Province

Zitieren

APA

Zhang, Y., Zhou, T., Li, Y., Xu, Y., Li, T., & Wang, J. (2026). Writing creativity, cohesion, and formal linguistic competence in LLMs: A comparative evaluation based on English and Chinese continuation writing. PLOS One, 21(6), e0335185. https://doi.org/10.1371/journal.pone.0335185

Vancouver

Zhang Y, Zhou T, Li Y, Xu Y, Li T, Wang J. Writing creativity, cohesion, and formal linguistic competence in LLMs: A comparative evaluation based on English and Chinese continuation writing. PLOS One. 2026;21(6):e0335185. doi:10.1371/journal.pone.0335185

BibTeX
@article{Zhang2026,
  author = {Zhang, Yao and Zhou, Tongquan and Li, Yulu and Xu, Yizhong and Li, Taohui and Wang, Juan},
  title = {Writing creativity, cohesion, and formal linguistic competence in LLMs: A comparative evaluation based on English and Chinese continuation writing},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {6},
  pages = {e0335185},
  doi = {10.1371/journal.pone.0335185}
}
Kostenlos starten →