GPT-4 generates accurate and readable patient education materials aligned with current oncological guidelines: A randomized assessment
GPT-4 erzeugt genaue und gut lesbare Patienteninformationsmaterialien im Einklang mit aktuellen onkologischen Leitlinien: randomisierte Bewertung
PLOS One · 04.06.2025 · 20 (6) · e0324175
DOI: 10.1371/journal.pone.0324175 ↗Auf einen Blick
GPT-4 erstellte aus EAU-Leitlinien schnell Patienteninformationsmaterialien mit besserer Lesbarkeit als die Originaltexte. Genauigkeit, Vollständigkeit und Klarheit waren vergleichbar.
Fragestellung
Kann generative KI lesbare, genaue und aktuelle Patienteninformationsmaterialien auf Basis onkologischer Leitlinien erstellen?
Methode
Einfach verblindete Online-Randomisierungsstudie mit 32 Mitgliedern der Young Academic Urologists; Bewertung originaler vs. GPT-4-generierter Materialien sowie Übersetzungen ins Niederländische, Französische, Deutsche, Italienische und Spanische.
Ergebnisse
Die Erstellung dauerte im Mittel 52,1 Sekunden. FKRE 43,5 vs. 70,8 und FKGL 11,6 vs. 6,1 zugunsten der GPT-4-Texte; p < .001. Keine Unterschiede in Genauigkeit, Vollständigkeit und Klarheit bei lokalisierten oder metastasierten urologischen Karzinomen.
Schlussfolgerung der Autor:innen
GPT-4-generierte Patientenmaterialien hatten bessere Lesbarkeit bei ähnlicher Genauigkeit, Vollständigkeit und Klarheit. Mit menschlicher Kontrolle kann GenAI die Aktualisierung und Erstellung solcher Materialien erleichtern.
Kernaussagen
- GPT-4-Text war leichter lesbar als der Originaltext
- Genauigkeit, Vollständigkeit und Klarheit waren ähnlich
- Übersetzungen wurden in 77,5 % als treu und in 67,5 % als klar bewertet
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Randomisierte kontrollierte Studie in der Zeitschrift PLOS One, online erschienen am 04.06.2025. Verfasst von Severin Rodler und 22 weiteren Autor:innen. Beteiligte Einrichtung: USC Institute of Urology and Catherine and Joseph Aresty Department of Urology, University of Southern California, Los Angeles, California, United St…. Hauptthemen laut MEDLINE: Patient Education as Topic, Practice Guidelines as Topic. Die Arbeit wurde bisher 34-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction and aim Guideline-based patient educational materials (PEMs) empower patients and reduce misinformation, but require frequent updates and must be adapted to the readability level of patients. The aim is to assess whether generative artificial intelligence (GenAI) can provide readable, accurate, and up-to-date PEMs that can be subsequently translated into multiple languages for broad dissemination. Study design and methods The European Association of Urology (EAU) guidelines for prostate, bladder, kidney, and testicular cancer were used as the knowledge base for GPT-4 to generate PEMs. Additionally, the PEMs were translated into five commonly spoken languages within the European Union (EU). The study was conducted through a single-blinded, online randomized assessment survey. After an initial pilot assessment of the GenAI-generated PEMs, thirty-two members of the Young Academic Urologists (YAU) groups evaluated the accuracy, completeness, and clarity of the original versus GPT-generated PEMs. The translation assessment involved two native speakers from different YAU groups for each language: Dutch, French, German, Italian, and Spanish. The primary outcomes were readability, accuracy, completeness, faithfulness, and clarity. Readability was measured using Flesch Kincaid Reading Ease (FKRE), Flesch Kincaid Grade Level (FKGL), Gunning Fog (GFS) scores and Smog (SI), Coleman Liau (CLI), Automated Readability (ARI) indexes. Accuracy, completeness, faithfulness, and clarity were rated on a 5-item Likert scale. Results The mean time to create layperson PEMs based on the latest guideline by GPT-4 was 52.1 seconds. The readability scores for the 8 original PEMs were lower than for the 8 GPT-4-generated PEMs (Mean FKRE: 43.5 vs. 70.8; p < .001). The required reading education levels were higher for original PEMs compared to GPT-4 generated PEMs (Mean FKGL: 11.6 vs. 6.1; p < .001). For all urological localized cancers, the original PEMs were not significantly different from the GPT-4 generated PEMs in accuracy, completeness, and clarity. Similarly, no differences were observed for metastatic cancers. Translations of GPT-generated PEMs were rated as faithful in 77.5% of cases and clear in 67.5% of cases. Conclusions and relevance GPT-4 generated PEMs have better readability levels compared to original PEMs while maintaining similar accuracy, completeness, and clarity. The use of GenAI’s information extraction and language capabilities, integrated with human oversight, can significantly reduce the workload and ensure up-to-date and accurate PEMs. Patient summary Some cancer facts made for patients can be hard to read or not in the right words for those with prostate, bladder, kidney, or testicular cancer. This study used AI to quickly make short and easy-to-read content from trusted facts. Doctors checked the AI content and found that they were just as accurate, complete, and clear as the original text made for patients. They also worked well in many languages. This AI tool can assist providers in making it easier for patients to understand their cancer and the best care they can get.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- USC Institute of Urology and Catherine and Joseph Aresty Department of Urology, University of Southern California, Los Angeles, California, United States of America.
- Artificial Intelligence Center at USC Urology, USC Institute of Urology, University of Southern California, Los Angeles, California, United States of America.
- Department of Urology, University Hospital Schleswig-Holstein, Campus Kiel, Kiel, Germany.
- Department of Urology, IRCCS San Raffaele Hospital and Vita-Salute San Raffaele University, Milan, Italy.
- Department of Surgery, Candiolo Cancer Institute, FPO-IRCCS, Turin, Italy.
- Department of Urology, Onze-Lieve-Vrouwziekenhuis Hospital, Aalst, Belgium.
ORCID-Profile: Mark Taratkin, Stefano Puliatti, Loïc Baekelandt, Riccardo Campi, Giovanni E Cacciamani
Zitieren
APA
Rodler, S., Cei, F., Ganjavi, C., Checcucci, E., De Backer, P., Rivero Belenchon, I., Taratkin, M., Puliatti, S., Veccia, A., Piazza, P., Baekelandt, L., Kowalewski, K. F., Gómez Rivas, J., Fankhauser, C. D., Moschini, M., Gandaglia, G., Campi, R., De Castro Abreu, A., Russo, G. I., …, & YAU Collaborators (2025). GPT-4 generates accurate and readable patient education materials aligned with current oncological guidelines: A randomized assessment. PLOS One, 20(6), e0324175. https://doi.org/10.1371/journal.pone.0324175
Vancouver
Rodler S, Cei F, Ganjavi C, Checcucci E, De Backer P, Rivero Belenchon I, et al. GPT-4 generates accurate and readable patient education materials aligned with current oncological guidelines: A randomized assessment. PLOS One. 2025;20(6):e0324175. doi:10.1371/journal.pone.0324175
BibTeX
@article{Rodler2025,
author = {Rodler, Severin and Cei, Francesco and Ganjavi, Conner and Checcucci, Enrico and De Backer, Pieter and Rivero Belenchon, Ines and Taratkin, Mark and Puliatti, Stefano and Veccia, Alessandro and Piazza, Pietro and Baekelandt, Loïc and Kowalewski, Karl-Friedrich and Gómez Rivas, Juan and Fankhauser, Christian D and Moschini, Marco and Gandaglia, Giorgio and Campi, Riccardo and De Castro Abreu, Andre and Russo, Giorgio I and Cocci, Andrea and Maruccia, Serena and Cacciamani, Giovanni E and YAU Collaborators, YAU Collaborators},
title = {GPT-4 generates accurate and readable patient education materials aligned with current oncological guidelines: A randomized assessment},
journal = {PLOS One},
year = {2025},
volume = {20},
number = {6},
pages = {e0324175},
doi = {10.1371/journal.pone.0324175}
}
EUVIMED