AI in Hand Surgery: Assessing Large Language Models in the Classification and Management of Hand Injuries
Journal of Clinical Medicine · 11.05.2024 · 13 (10) · 2832
DOI: 10.3390/jcm13102832 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Clinical Medicine, online erschienen am 11.05.2024. Verfasst von Sophia M Pressman und 4 weiteren Autor:innen. Beteiligte Einrichtung: Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
OpenAI’s ChatGPT (San Francisco, CA, USA) and Google’s Gemini (Mountain View, CA, USA) are two large language models that show promise in improving and expediting medical decision making in hand surgery. Evaluating the applications of these models within the field of hand surgery is warranted. This study aims to evaluate ChatGPT-4 and Gemini in classifying hand injuries and recommending treatment.
Methods
Gemini and ChatGPT were given 68 fictionalized clinical vignettes of hand injuries twice. The models were asked to use a specific classification system and recommend surgical or nonsurgical treatment. Classifications were scored based on correctness. Results were analyzed using descriptive statistics, a paired two-tailed t-test, and sensitivity testing.
Results
Gemini, correctly classifying 70.6% hand injuries, demonstrated superior classification ability over ChatGPT (mean score 1.46 vs. 0.87, p-value < 0.001). For management, ChatGPT demonstrated higher sensitivity in recommending surgical intervention compared to Gemini (98.0% vs. 88.8%), but lower specificity (68.4% vs. 94.7%). When compared to ChatGPT, Gemini demonstrated greater response replicability.
Conclusions
Large language models like ChatGPT and Gemini show promise in assisting medical decision making, particularly in hand surgery, with Gemini generally outperforming ChatGPT. These findings emphasize the importance of considering the strengths and limitations of different models when integrating them into clinical practice.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA.
- Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA.
ORCID-Profile: Sophia M Pressman, Sahar Borna, Cesar A Gomez-Cabello, Antonio Jorge Forte
Zitieren
APA
Pressman, S. M., Borna, S., Gomez-Cabello, C. A., Haider, S. A., & Forte, A. J. (2024). AI in Hand Surgery: Assessing Large Language Models in the Classification and Management of Hand Injuries. Journal of Clinical Medicine, 13(10), 2832. https://doi.org/10.3390/jcm13102832
Vancouver
Pressman SM, Borna S, Gomez-Cabello CA, Haider SA, Forte AJ. AI in Hand Surgery: Assessing Large Language Models in the Classification and Management of Hand Injuries. Journal of Clinical Medicine. 2024;13(10):2832. doi:10.3390/jcm13102832
BibTeX
@article{Pressman2024,
author = {Pressman, Sophia M and Borna, Sahar and Gomez-Cabello, Cesar A and Haider, Syed Ali and Forte, Antonio Jorge},
title = {AI in Hand Surgery: Assessing Large Language Models in the Classification and Management of Hand Injuries},
journal = {Journal of Clinical Medicine},
year = {2024},
volume = {13},
number = {10},
pages = {2832},
doi = {10.3390/jcm13102832}
}
EUVIMED