EUVIMEDEUVIMED entdecken
Crossref

Large Language Model Chat GPT-4 Can Outperform Clinicians in Endoscopy Triage

J. R. Campion, J. Cudmore, E. Keating, H. Kerr, S. Stewart, J. Leyden, B. Kelleher, J. Mulsow, C. Lahiff, G. Bennett

Endoscopy · 01.04.2024 · 56 (S 02 S 02) · S29-S30

DOI: 10.1055/s-0044-1782747 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Endoscopy, erschienen am 01.04.2024. Verfasst von J. R. Campion und 9 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aims

We sought to compare adherence to current national and international guidelines on triage of new patient referrals and surveillance endoscopy procedures between a commercially-available general large language model (LLM) and clinical staff working in an academic department of gastroenterology.

Methods

A consultant gastroenterologist used guidelines from the European Society of Gastrointestinal Endoscopy (ESGE), Ireland's Health Information and Quality Authority (HIQA) and National Cancer Control Programme (NCCP), the UK's National Institute for Health and Care Excellence (NICE) and the British Society of Gastroenterology (BSG) to construct 64 fictional patient cases requiring consideration for symptomatic or surveillance endoscopy. The cases were divided across five categories: lower gastrointestinal symptomatic (LGI), upper GI symptomatic (UGI), family history of colorectal carcinoma (FHCC), polyp surveillance (PS) and Barrett’s oesophagus surveillance (BS). Consultants, registrars and nurses in the Department of Gastroenterology and the general LLM, Chat GPT-4, were asked to read the cases and choose the appropriate disposition for each (attempt 1). Each participant was later given text or an algorithm summarising the relevant guidelines and asked to respond to each case again (attempt 2).

Results

Twenty clinicians and one LLM participated in the study. In attempt 1, the LLM median (IQR) score was higher than clinician in LGI [70 (60,70) vs 50 (37.5,60), p=0.008] and FHCC [82 (73,82) vs 36 (27,65), p=0.003] while there was no statistically significant difference in BS [71 (64,71) vs 57 (43,64), p=0.37)], PS [31 (23,46) vs 31 (21,48), p=0.84] or UGI [50 (50,62.5) vs 53 (50,57), p=0.81)]. In attempt 2, median clinician scores improved to LGI [80% (70,80)], FHCC [78% (52,91)], BS [79% (67,86)], PS [62% (42,71) and UGI [75% (74,81)]. After inputting guideline summaries, LLM performance improved in LGI (70%), PS (69%) and BS (71%) but worsened in UGI (50%) and FHCC (64%).

Conclusions

LLMs may prove a useful tool for administrative and triage tasks for busy clinicians, but unsupervised decision-making cannot yet be delegated to LLMs. Specific models trained only on discrete inputs including relevant guidelines may improve performance. Clinicians using LLMs will require specific training on how best to converse with LLMs, to ensure optimal outcomes for patients and health services. Publication History Article published online: 15 April 2024 © 2024. European Society of Gastrointestinal Endoscopy. All rights reserved. Georg Thieme Verlag KG Rüdigerstraße 14, 70469 Stuttgart, Germany

Zitieren

APA

Campion, J. R., Cudmore, J., Keating, E., Kerr, H., Stewart, S., Leyden, J., Kelleher, B., Mulsow, J., Lahiff, C., & Bennett, G. (2024). Large Language Model Chat GPT-4 Can Outperform Clinicians in Endoscopy Triage. Endoscopy, 56(S 02 S 02), S29-S30. https://doi.org/10.1055/s-0044-1782747

Vancouver

Campion JR, Cudmore J, Keating E, Kerr H, Stewart S, Leyden J, et al. Large Language Model Chat GPT-4 Can Outperform Clinicians in Endoscopy Triage. Endoscopy. 2024;56(S 02 S 02):S29-S30. doi:10.1055/s-0044-1782747

BibTeX
@article{Campion2024,
  author = {Campion, J. R. and Cudmore, J. and Keating, E. and Kerr, H. and Stewart, S. and Leyden, J. and Kelleher, B. and Mulsow, J. and Lahiff, C. and Bennett, G.},
  title = {Large Language Model Chat GPT-4 Can Outperform Clinicians in Endoscopy Triage},
  journal = {Endoscopy},
  year = {2024},
  volume = {56},
  number = {S 02 S 02},
  pages = {S29--S30},
  doi = {10.1055/s-0044-1782747}
}
Kostenlos starten →