Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0297701 ↗Kurz gefasst
Randomisierte kontrollierte Studie in der Zeitschrift PLOS ONE, 2024. Verfasst von Taesoon Hwang und 7 weiteren Autor:innen. Beteiligte Einrichtung: Warwick Medical School, University of Warwick, Coventry, United Kingdom. Hauptthemen laut MEDLINE: Periodicals as Topic. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
ChatGPT, a sophisticated large language model (LLM), has garnered widespread attention for its ability to mimic human-like communication. As recent studies indicate a potential supportive role of ChatGPT in academic writing, we assessed the LLM’s capacity to generate accurate and comprehensive scientific abstracts from published Randomised Controlled Trial (RCT) data, focusing on the adherence to the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) statement, in comparison to the original authors’ abstracts. Methodology RCTs, identified in a PubMed/MEDLINE search post-September 2021 across various medical disciplines, were subjected to abstract generation via ChatGPT versions 3.5 and 4, following the guidelines of the respective journals. The overall quality score (OQS) of each abstract was determined by the total number of adequately reported components from the 18-item CONSORT-A checklist. Additional outcome measures included percent adherence to each CONOSORT-A item, readability, hallucination rate, and regression analysis of reporting quality determinants.
Results
Original abstracts achieved a mean OQS of 11.89 (95% CI: 11.23–12.54), outperforming GPT 3.5 (7.89; 95% CI: 7.32–8.46) and GPT 4 (5.18; 95% CI: 4.64–5.71). Compared to GPT 3.5 and 4 outputs, original abstracts were more adherent with 10 and 14 CONSORT-A items, respectively. In blind assessments, GPT 3.5-generated abstracts were deemed most readable in 62.22% of cases which was significantly greater than the original (31.11%; P = 0.003) and GPT 4-generated (6.67%; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Warwick Medical School, University of Warwick, Coventry, United Kingdom.
- University Hospitals Coventry and Warwickshire, Coventry, United Kingdom.
- Royal Devon and Exeter Hospital, Exeter, United Kingdom.
ORCID-Profile: Taesoon Hwang, Thomas Roberts
Zitieren
APA
Hwang, T., Aggarwal, N., Khan, P. Z., Roberts, T., Mahmood, A., Griffiths, M. M., Parsons, N., & Khan, S. (2024). Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts. PLOS ONE. https://doi.org/10.1371/journal.pone.0297701
Vancouver
Hwang T, Aggarwal N, Khan PZ, Roberts T, Mahmood A, Griffiths MM, et al. Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts. PLOS ONE. 2024. doi:10.1371/journal.pone.0297701
BibTeX
@article{Hwang2024,
author = {Hwang, Taesoon and Aggarwal, Nishant and Khan, Pir Zarak and Roberts, Thomas and Mahmood, Amir and Griffiths, Madlen M and Parsons, Nick and Khan, Saboor},
title = {Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0297701}
}
EUVIMED