EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Using AI-generated suggestions from ChatGPT to optimize clinical decision support

Siru Liu, Aileen P Wright, Barron L Patterson, Jonathan P Wanderer, Robert W Turer iD, Scott D Nelson iD, Allison B McCoy iD, Dean F Sittig iD, Adam Wright iD

Journal of the American Medical Informatics Association · 20.06.2023 · 30 (7) · 1237-1245

DOI: 10.1093/jamia/ocad072 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 22.04.2023. Verfasst von Siru Liu und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA. Hauptthemen laut MEDLINE: Decision Support Systems, Clinical, Learning Health System. Die Arbeit wurde bisher 401-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NLM NIH HHS, NIA NIH HHS, NIH.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To determine if ChatGPT can generate useful suggestions for improving clinical decision support (CDS) logic and to assess noninferiority compared to human-generated suggestions.

Methods

We supplied summaries of CDS logic to ChatGPT, an artificial intelligence (AI) tool for question answering that uses a large language model, and asked it to generate suggestions. We asked human clinician reviewers to review the AI-generated suggestions as well as human-generated suggestions for improving the same CDS alerts, and rate the suggestions for their usefulness, acceptance, relevance, understanding, workflow, bias, inversion, and redundancy.

Results

Five clinicians analyzed 36 AI-generated suggestions and 29 human-generated suggestions for 7 alerts. Of the 20 suggestions that scored highest in the survey, 9 were generated by ChatGPT. The suggestions generated by AI were found to offer unique perspectives and were evaluated as highly understandable and relevant, with moderate usefulness, low acceptance, bias, inversion, redundancy.

Conclusion

AI-generated suggestions could be an important complementary part of optimizing CDS alerts, can identify potential improvements to alert logic and support their implementation, and may even be able to assist experts in formulating their own suggestions for CDS improvement. ChatGPT shows great potential for using large language models and reinforcement learning from human feedback to improve CDS alert logic and potentially other medical areas involving complex, clinical logic, a key step in the development of an advanced learning health system.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Decision Support Systems, ClinicalLearning Health SystemHumansArtificial IntelligenceLanguageWorkflow

Stichwörter der Autor:innen

artificial intelligenceclinical decision supportlarge language model

Autor:innen und Institutionen

  • Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA.
  • Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA.
  • Department of Pediatrics, Vanderbilt University Medical Center, Nashville, Tennessee, USA.
  • Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, Tennessee, USA.
  • Department of Emergency Medicine, University of Texas Southwestern Medical Center, Dallas, Texas, USA.
  • Clinical Informatics Center, University of Texas Southwestern Medical Center, Dallas, Texas, USA.

ORCID-Profile: Robert W Turer, Scott D Nelson, Allison B McCoy, Dean F Sittig, Adam Wright

Förderung

  • NLM NIH HHS
  • NIA NIH HHS
  • NIH

Zitieren

APA

Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023). Using AI-generated suggestions from ChatGPT to optimize clinical decision support. Journal of the American Medical Informatics Association, 30(7), 1237-1245. https://doi.org/10.1093/jamia/ocad072

Vancouver

Liu S, Wright AP, Patterson BL, Wanderer JP, Turer RW, Nelson SD, et al. Using AI-generated suggestions from ChatGPT to optimize clinical decision support. Journal of the American Medical Informatics Association. 2023;30(7):1237-1245. doi:10.1093/jamia/ocad072

BibTeX
@article{Liu2023,
  author = {Liu, Siru and Wright, Aileen P and Patterson, Barron L and Wanderer, Jonathan P and Turer, Robert W and Nelson, Scott D and McCoy, Allison B and Sittig, Dean F and Wright, Adam},
  title = {Using AI-generated suggestions from ChatGPT to optimize clinical decision support},
  journal = {Journal of the American Medical Informatics Association},
  year = {2023},
  volume = {30},
  number = {7},
  pages = {1237--1245},
  doi = {10.1093/jamia/ocad072}
}
Kostenlos starten →