Can ChatGPT write better scientific titles? A comparative evaluation of human-written and AI-generated titles
F1000Research · 2026
DOI: 10.12688/f1000research.173647.2 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift F1000Research, 2026. Verfasst von Paul Sebo und 2 weiteren Autor:innen. Beteiligte Einrichtung: University Institute for Primary Care, University of Geneva, Geneva, Switzerland. Hauptthemen laut MEDLINE: Writing. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
Large language models (LLMs) such as GPT-4 are increasingly used in scientific writing, yet little is known about how AI-generated scientific titles are perceived by researchers in terms of quality.
Objective
To compare the perceived alignment with the abstract content (as a surrogate for perceived accuracy), appeal, and overall preference for AI-generated versus human-written scientific titles.
Methods
We conducted a blinded comparative study with 21 researchers from diverse academic backgrounds. A random sample of 50 original titles was selected from 10 high-impact general internal medicine journals. For each title, an alternative version was generated using GPT-4.0. Each rater evaluated 50 pairs of titles, each pair consisting of one original and one AI-generated version, without knowing the source of the titles or the purpose of the study. For each pair, raters independently assessed both titles on perceived alignment with the abstract content and appeal, and indicated their overall preference. We analyzed alignment and appeal using Wilcoxon signed-rank tests and mixed-effects ordinal logistic regressions, preferences using McNemar’s test and mixed-effects logistic regression, and inter-rater agreement with Gwet’s AC.
Results
AI-generated titles received significantly higher ratings for both perceived alignment with the abstract content (mean 7.9 vs. 6.7, p-value
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- University Institute for Primary Care, University of Geneva, Geneva, Switzerland.
- Zhejiang Tongji Vocational College of Science and Technology, Hangzhou, Zhejiang, China.
- School of Library and Information Management, Emporia State University, Emporia, Kansas, USA.
ORCID-Profile: Paul Sebo
Zitieren
APA
Sebo, P., Nie, B., & Wang, T. (2026). Can ChatGPT write better scientific titles? A comparative evaluation of human-written and AI-generated titles. F1000Research. https://doi.org/10.12688/f1000research.173647.2
Vancouver
Sebo P, Nie B, Wang T. Can ChatGPT write better scientific titles? A comparative evaluation of human-written and AI-generated titles. F1000Research. 2026. doi:10.12688/f1000research.173647.2
BibTeX
@article{Sebo2026,
author = {Sebo, Paul and Nie, Bing and Wang, Ting},
title = {Can ChatGPT write better scientific titles? A comparative evaluation of human-written and AI-generated titles},
journal = {F1000Research},
year = {2026},
doi = {10.12688/f1000research.173647.2}
}
EUVIMED