EUVIMEDEUVIMED entdecken
Randomisierte kontrollierte StudieOpen Access · CC BY 4.0

Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts

Taesoon Hwang iD, Nishant Aggarwal, Pir Zarak Khan, Thomas Roberts iD, Amir Mahmood, Madlen M Griffiths, Nick Parsons, Saboor Khan

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0297701 ↗

Kurz gefasst

Randomisierte kontrollierte Studie in der Zeitschrift PLOS ONE, 2024. Verfasst von Taesoon Hwang und 7 weiteren Autor:innen. Beteiligte Einrichtung: Warwick Medical School, University of Warwick, Coventry, United Kingdom. Hauptthemen laut MEDLINE: Periodicals as Topic. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

ChatGPT, a sophisticated large language model (LLM), has garnered widespread attention for its ability to mimic human-like communication. As recent studies indicate a potential supportive role of ChatGPT in academic writing, we assessed the LLM’s capacity to generate accurate and comprehensive scientific abstracts from published Randomised Controlled Trial (RCT) data, focusing on the adherence to the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) statement, in comparison to the original authors’ abstracts. Methodology RCTs, identified in a PubMed/MEDLINE search post-September 2021 across various medical disciplines, were subjected to abstract generation via ChatGPT versions 3.5 and 4, following the guidelines of the respective journals. The overall quality score (OQS) of each abstract was determined by the total number of adequately reported components from the 18-item CONSORT-A checklist. Additional outcome measures included percent adherence to each CONOSORT-A item, readability, hallucination rate, and regression analysis of reporting quality determinants.

Results

Original abstracts achieved a mean OQS of 11.89 (95% CI: 11.23–12.54), outperforming GPT 3.5 (7.89; 95% CI: 7.32–8.46) and GPT 4 (5.18; 95% CI: 4.64–5.71). Compared to GPT 3.5 and 4 outputs, original abstracts were more adherent with 10 and 14 CONSORT-A items, respectively. In blind assessments, GPT 3.5-generated abstracts were deemed most readable in 62.22% of cases which was significantly greater than the original (31.11%; P = 0.003) and GPT 4-generated (6.67%; P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Periodicals as TopicHumansWritingReference StandardsPublicationsHallucinations

Autor:innen und Institutionen

  • Warwick Medical School, University of Warwick, Coventry, United Kingdom.
  • University Hospitals Coventry and Warwickshire, Coventry, United Kingdom.
  • Royal Devon and Exeter Hospital, Exeter, United Kingdom.

ORCID-Profile: Taesoon Hwang, Thomas Roberts

Zitieren

APA

Hwang, T., Aggarwal, N., Khan, P. Z., Roberts, T., Mahmood, A., Griffiths, M. M., Parsons, N., & Khan, S. (2024). Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts. PLOS ONE. https://doi.org/10.1371/journal.pone.0297701

Vancouver

Hwang T, Aggarwal N, Khan PZ, Roberts T, Mahmood A, Griffiths MM, et al. Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts. PLOS ONE. 2024. doi:10.1371/journal.pone.0297701

BibTeX
@article{Hwang2024,
  author = {Hwang, Taesoon and Aggarwal, Nishant and Khan, Pir Zarak and Roberts, Thomas and Mahmood, Amir and Griffiths, Madlen M and Parsons, Nick and Khan, Saboor},
  title = {Can ChatGPT assist authors with abstract writing in medical journals? Evaluating the quality of scientific abstracts generated by ChatGPT and original abstracts},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0297701}
}
Kostenlos starten →