Completeness and readability of GPT-4-generated multilingual discharge instructions in the pediatric emergency department
JAMIA Open · 01.07.2024 · 7 (3)
DOI: 10.1093/jamiaopen/ooae050 ↗Kurz gefasst
Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 01.07.2024. Verfasst von Alex Gimeno und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, TN 37232, United States. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Wellcome Leap MCPsych.
Abstract (Originalsprache)
Objectives
The aim of this study was to assess the completeness and readability of generative pre-trained transformer-4 (GPT-4)-generated discharge instructions at prespecified reading levels for common pediatric emergency room complaints. Materials and Methods The outputs for 6 discharge scenarios stratified by reading level (fifth or eighth grade) and language (English, Spanish) were generated fivefold using GPT-4. Specifically, 120 discharge instructions were produced and analyzed (6 scenarios: 60 in English, 60 in Spanish; 60 at a fifth-grade reading level, 60 at an eighth-grade reading level) and compared for completeness and readability (between language, between reading level, and stratified by group and reading level). Completeness was defined as the proportion of literature-derived key points included in discharge instructions. Readability was quantified using Flesch-Kincaid (English) and Fernandez-Huerta (Spanish) readability scores.
Results
English-language GPT-generated discharge instructions contained a significantly higher proportion of must-include discharge instructions than those in Spanish (English: mean (standard error of the mean) = 62% (3%), Spanish: 53% (3%), P = .02). In the fifth-grade and eighth-grade level conditions, there was no significant difference between English and Spanish outputs in completeness. Readability did not differ across languages.
Discussion
GPT-4 produced readable discharge instructions in English and Spanish while modulating document reading level. Discharge instructions in English tended to have higher completeness than those in Spanish.
Conclusion
Future research in prompt engineering and GPT-4 performance, both generally and in multiple languages, is needed to reduce potential for health disparities by language and reading level.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, TN 37232, United States.
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
- Department of Psychiatry and Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
ORCID-Profile: Alex Gimeno, Kevin Krause
Förderung
- Wellcome Leap MCPsych
Zitieren
APA
Gimeno, A., Krause, K., D'Souza, S., & Walsh, C. G. (2024). Completeness and readability of GPT-4-generated multilingual discharge instructions in the pediatric emergency department. JAMIA Open, 7(3). https://doi.org/10.1093/jamiaopen/ooae050
Vancouver
Gimeno A, Krause K, D'Souza S, Walsh CG. Completeness and readability of GPT-4-generated multilingual discharge instructions in the pediatric emergency department. JAMIA Open. 2024;7(3). doi:10.1093/jamiaopen/ooae050
BibTeX
@article{Gimeno2024,
author = {Gimeno, Alex and Krause, Kevin and D'Souza, Starina and Walsh, Colin G},
title = {Completeness and readability of GPT-4-generated multilingual discharge instructions in the pediatric emergency department},
journal = {JAMIA Open},
year = {2024},
volume = {7},
number = {3},
doi = {10.1093/jamiaopen/ooae050}
}
EUVIMED