EUVIMEDEUVIMED entdecken
Randomisierte kontrollierte StudieOpen Access

Patient-Representing Population's Perceptions of GPT-Generated Versus Standard Emergency Department Discharge Instructions: Randomized Blind Survey Assessment

Thomas Huang iD, Conrad Safranek iD, Vimig Socrates iD, David Chartash iD, Donald Wright iD, Monisha Dilip iD, Rohit B Sangal iD, Richard Andrew Taylor iD

Journal of Medical Internet Research · 02.08.2024 · 26 · e60336

DOI: 10.2196/60336 ↗

Kurz gefasst

Randomisierte kontrollierte Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 02.08.2024. Verfasst von Thomas Huang und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Emergency Medicine, Yale School of Medicine, New Haven, CT, United States. Hauptthemen laut MEDLINE: Emergency Service, Hospital, Patient Discharge. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NHLBI NIH HHS, NCATS NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Discharge instructions are a key form of documentation and patient communication in the time of transition from the emergency department (ED) to home. Discharge instructions are time-consuming and often underprioritized, especially in the ED, leading to discharge delays and possibly impersonal patient instructions. Generative artificial intelligence and large language models (LLMs) offer promising methods of creating high-quality and personalized discharge instructions; however, there exists a gap in understanding patient perspectives of LLM-generated discharge instructions.

Objective

We aimed to assess the use of LLMs such as ChatGPT in synthesizing accurate and patient-accessible discharge instructions in the ED.

Methods

We synthesized 5 unique, fictional ED encounters to emulate real ED encounters that included a diverse set of clinician history, physical notes, and nursing notes. These were passed to GPT-4 in Azure OpenAI Service (Microsoft) to generate LLM-generated discharge instructions. Standard discharge instructions were also generated for each of the 5 unique ED encounters. All GPT-generated and standard discharge instructions were then formatted into standardized after-visit summary documents. These after-visit summaries containing either GPT-generated or standard discharge instructions were randomly and blindly administered to Amazon MTurk respondents representing patient populations through Amazon MTurk Survey Distribution. Discharge instructions were assessed based on metrics of interpretability of significance, understandability, and satisfaction.

Results

Our findings revealed that survey respondents’ perspectives regarding GPT-generated and standard discharge instructions were significantly (P=.01) more favorable toward GPT-generated return precautions, and all other sections were considered noninferior to standard discharge instructions. Of the 156 survey respondents, GPT-generated discharge instructions were assigned favorable ratings, “agree” and “strongly agree,” more frequently along the metric of interpretability of significance in discharge instruction subsections regarding diagnosis, procedures, treatment, post-ED medications or any changes to medications, and return precautions. Survey respondents found GPT-generated instructions to be more understandable when rating procedures, treatment, post-ED medications or medication changes, post-ED follow-up, and return precautions. Satisfaction with GPT-generated discharge instruction subsections was the most favorable in procedures, treatment, post-ED medications or medication changes, and return precautions. Wilcoxon rank-sum test of Likert responses revealed significant differences (P=.01) in the interpretability of significant return precautions in GPT-generated discharge instructions compared to standard discharge instructions but not for other evaluation metrics and discharge instruction subsections.

Conclusions

This study demonstrates the potential for LLMs such as ChatGPT to act as a method of augmenting current documentation workflows in the ED to reduce the documentation burden of physicians. The ability of LLMs to provide tailored instructions for patients by improving readability and making instructions more applicable to patients could improve upon the methods of communication that currently exist.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Emergency Service, HospitalPatient DischargeHumansFemaleMaleSurveys and QuestionnairesAdultMiddle AgedArtificial Intelligence

Stichwörter der Autor:innen

ChatGPTartificial intelligencedischarge instructionsemergency departmentemergency medicinelarge language modelsmachine learningnatural language processingsurveys and questionaries

Autor:innen und Institutionen

  • Department of Emergency Medicine, Yale School of Medicine, New Haven, CT, United States.
  • Department for Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, United States.
  • Program of Computational Biology and Bioinformatics, Yale University, New Haven, CT, United States.
  • School of Medicine, University College Dublin, National University of Ireland, Dublin, Ireland.

ORCID-Profile: Thomas Huang, Conrad Safranek, Vimig Socrates, David Chartash, Donald Wright, Monisha Dilip, Rohit B Sangal, Richard Andrew Taylor

Förderung

  • NHLBI NIH HHS
  • NCATS NIH HHS

Zitieren

APA

Huang, T., Safranek, C., Socrates, V., Chartash, D., Wright, D., Dilip, M., Sangal, R. B., & Taylor, R. A. (2024). Patient-Representing Population's Perceptions of GPT-Generated Versus Standard Emergency Department Discharge Instructions: Randomized Blind Survey Assessment. Journal of Medical Internet Research, 26, e60336. https://doi.org/10.2196/60336

Vancouver

Huang T, Safranek C, Socrates V, Chartash D, Wright D, Dilip M, et al. Patient-Representing Population's Perceptions of GPT-Generated Versus Standard Emergency Department Discharge Instructions: Randomized Blind Survey Assessment. Journal of Medical Internet Research. 2024;26:e60336. doi:10.2196/60336

BibTeX
@article{Huang2024,
  author = {Huang, Thomas and Safranek, Conrad and Socrates, Vimig and Chartash, David and Wright, Donald and Dilip, Monisha and Sangal, Rohit B and Taylor, Richard Andrew},
  title = {Patient-Representing Population's Perceptions of GPT-Generated Versus Standard Emergency Department Discharge Instructions: Randomized Blind Survey Assessment},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  volume = {26},
  pages = {e60336},
  doi = {10.2196/60336}
}
Kostenlos starten →