Would doctors dream of electric blood bankers? Large language model‐based artificial intelligence performs well in many aspects of transfusion medicine
Transfusion · 2023
DOI: 10.1111/trf.17526 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transfusion, 2023. Verfasst von Nathan C Hurley und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Anesthesiology, University of Wisconsin-Madison, Madison, Wisconsin, USA. Hauptthemen laut MEDLINE: Transfusion Medicine, Physicians. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Background
Large language models (LLMs) excel at answering knowledge‐based questions. Many aspects of blood banking and transfusion medicine involve no direct patient care and require only knowledge and judgment. We hypothesized that public LLMs could perform such tasks with accuracy and precision. Study Design and Methods We presented three sets of tasks to three publicly‐available LLMs (Bard, GPT‐3.5, and GPT‐4). The first was to review short case presentations and then decide if a red blood cell transfusion was indicated. The second task was to answer a set of consultation questions common in clinical transfusion practice. The third task was to take a multiple‐choice test experimentally validated to assess internal medicine postgraduate knowledge of transfusion practice (the BEST‐TEST).
Results
In the first task, the area under the receiver operating characteristic curve for correct transfusion decisions was 0.65, 0.90, and 0.92, respectively for Bard, GPT‐3.5 and GPT‐4. All three models had a modest rate of acceptable responses to the consultation questions. Average scores on the BEST‐TEST were 55%, 40%, and 87%, respectively.
Conclusion
When presented with transfusion medicine tasks in natural language, publicly available LLMs demonstrated a range of ability, but GPT‐4 consistently scored very well in all tasks. Research is needed to assess the utility of LLMs in transfusion medicine practice. Transfusion Medicine physicians should consider their role alongside such technologies, and how they might be used for the benefit and safety of patients.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Anesthesiology, University of Wisconsin-Madison, Madison, Wisconsin, USA.
- Department of Pathology & Laboratory Medicine, University of Wisconsin-Madison, Madison, Wisconsin, USA.
ORCID-Profile: Nathan C Hurley, Aaron S Hess
Zitieren
APA
Hurley, N. C., Schroeder, K. M., & Hess, A. S. (2023). Would doctors dream of electric blood bankers? Large language model‐based artificial intelligence performs well in many aspects of transfusion medicine. Transfusion. https://doi.org/10.1111/trf.17526
Vancouver
Hurley NC, Schroeder KM, Hess AS. Would doctors dream of electric blood bankers? Large language model‐based artificial intelligence performs well in many aspects of transfusion medicine. Transfusion. 2023. doi:10.1111/trf.17526
BibTeX
@article{Hurley2023,
author = {Hurley, Nathan C and Schroeder, Kristopher M and Hess, Aaron S},
title = {Would doctors dream of electric blood bankers? Large language model‐based artificial intelligence performs well in many aspects of transfusion medicine},
journal = {Transfusion},
year = {2023},
doi = {10.1111/trf.17526}
}
EUVIMED