EUVIMEDEUVIMED entdecken
EvaluationsstudieOpen Access

Using ChatGPT-4 for Lay Summarization in Prostate Cancer Research to Advance Patient-Centered Communication: Large-Scale Generative AI Performance Evaluation

Emily Rinderknecht, Simon U Engelmann, Veronika Saberi, Clemens Kirschner, Anton P Kravchuk, Anna Schmelzer, Johannes Breyer, Christopher Goßler, Roman Mayr, Christian Gilfrich, Maximilian Burger, Dominik von Winning und 5 weitere

Journal of Medical Internet Research · 2025

DOI: 10.2196/76598 ↗

Kurz gefasst

Evaluationsstudie in der Zeitschrift Journal of Medical Internet Research, 2025. Verfasst von Emily Rinderknecht und 16 weiteren Autor:innen. Beteiligte Einrichtung: Working Group on Artificial Intelligence and Digitalization of the German Society of Urology, Germany. Hauptthemen laut MEDLINE: Prostatic Neoplasms, Generative Artificial Intelligence, Patient Education as Topic, Plain Language Summaries. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The increasing volume and complexity of biomedical literature pose challenges for making scientific knowledge accessible to lay audiences. Lay summaries, now widely encouraged or required by journals, aim to bridge this gap by promoting health literacy, patient engagement, and public trust. However, many are written by scientists without formal training in plain-language communication, often resulting in limited clarity, readability, and consistency. Generative large language models such as ChatGPT-4 offer a scalable opportunity to support lay summary creation, though their effectiveness within specific clinical domains has not been systematically evaluated at scale.

Objective

This study aimed to assess ChatGPT-4’s performance in generating lay summaries for prostate cancer studies. A secondary objective was to evaluate how prompt design influences summary quality, aiming to provide practical guidance for the use of generative artificial intelligence (AI) in scientific publishing.

Methods

A total of 204 consecutive articles on prostate cancer were extracted from a high-ranking oncology journal mandating lay summaries. Each abstract was processed with ChatGPT-4 using 2 prompts: a simple prompt based on the journal’s guidelines and an extended prompt refined to improve readability. AI-generated and original summaries were evaluated using 3 criteria: readability (Flesch-Kincaid Reading Ease [FKRE]), factual accuracy (5-point Likert scale, blinded rating by 2 clinical experts), and compliance with word count instructions (120‐150 words). Summaries were classified as high-quality as a composite outcome if they met all 3 benchmarks: FKRE >30, accuracy ≥4 from both raters, and word count within range. Statistical comparisons used Wilcoxon signed-rank and paired 2-tailed t tests ( P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Prostatic NeoplasmsGenerative Artificial IntelligencePatient Education as TopicPlain Language SummariesCommunicationHumansMalePatient-Centered CareComprehensionPeriodicals as Topic

Stichwörter der Autor:innen

artificial intelligence in publishingcancer information accessibilitydigital health communicationhealth literacyhuman-AI collaborationlarge language modelsnatural language generationpatient engagementprompt engineeringreadability assessment

Autor:innen und Institutionen

  • Working Group on Artificial Intelligence and Digitalization of the German Society of Urology, Germany.
  • Department of Urology, University of Regensburg, Caritas St Josef Medical Center, Landshuter Street 65, Regensburg, 93053, Germany, 49 9417821000.
  • Department of Urology, St. Elisabeth Hospital Straubing, Straubing, Germany.
  • Department of Urology, Nuremberg General Hospital, Paracelsus Medical University, Nuremberg, Germany.
  • Department of Urology, University Hospital Augsburg, Augsburg, Germany.
  • Department of Urology, Faculty of Health Sciences Brandenburg, Brandenburg Medical School Theodor Fontan, Brandenburg, Germany.

Zitieren

APA

Rinderknecht, E., Engelmann, S. U., Saberi, V., Kirschner, C., Kravchuk, A. P., Schmelzer, A., Breyer, J., Goßler, C., Mayr, R., Gilfrich, C., Burger, M., von Winning, D., Borgmann, H., Wülfing, C., Merseburger, A. S., Haas, M., & May, M. (2025). Using ChatGPT-4 for Lay Summarization in Prostate Cancer Research to Advance Patient-Centered Communication: Large-Scale Generative AI Performance Evaluation. Journal of Medical Internet Research. https://doi.org/10.2196/76598

Vancouver

Rinderknecht E, Engelmann SU, Saberi V, Kirschner C, Kravchuk AP, Schmelzer A, et al. Using ChatGPT-4 for Lay Summarization in Prostate Cancer Research to Advance Patient-Centered Communication: Large-Scale Generative AI Performance Evaluation. Journal of Medical Internet Research. 2025. doi:10.2196/76598

BibTeX
@article{Rinderknecht2025,
  author = {Rinderknecht, Emily and Engelmann, Simon U and Saberi, Veronika and Kirschner, Clemens and Kravchuk, Anton P and Schmelzer, Anna and Breyer, Johannes and Goßler, Christopher and Mayr, Roman and Gilfrich, Christian and Burger, Maximilian and von Winning, Dominik and Borgmann, Hendrik and Wülfing, Christian and Merseburger, Axel S and Haas, Maximilian and May, Matthias},
  title = {Using ChatGPT-4 for Lay Summarization in Prostate Cancer Research to Advance Patient-Centered Communication: Large-Scale Generative AI Performance Evaluation},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  doi = {10.2196/76598}
}
Kostenlos starten →