EUVIMEDEUVIMED entdecken
Crossref

Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis

Asaf Levartovsky, Lior Dar, Shmuel Fay, Offir Ukashi, Eyal Shachar, Tal Engel, Shomron Ben-Horin, Edoardo V Savarino, Bella Ungar

Digestive and Liver Disease · 2025

DOI: 10.1016/j.dld.2025.11.009 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Digestive and Liver Disease, 2025. Verfasst von Asaf Levartovsky und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Gastroenterology, Sheba Medical Center, Israel; Gray Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel. Electronic address: al…. Hauptthemen laut MEDLINE: Eosinophilic Esophagitis, Gastroenterologists, Gastroenterology, Clinical Competence. Die Arbeit nennt 31 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: The comprehensive evaluation of multimodal large language models (LLMs) in gastroenterological image-analysis remains limited. We aimed to assess the accuracy of the Eosinophilic Esophagitis Endoscopic Reference Score (EREFS) scoring system for Eosinophilic Esophagitis (EoE) across gastroenterology (GI) clinicians with varying levels of experience. METHODS: Fifty real-world endoscopic images of EoE patients were graded based on the original EREFS score by a gold standard anchor-rater. EREFS gradings of GI specialists, fellows and three multimodal LLMs (ChatGPT-4o, Claude Sonnet 3.5, Perplexity Sonar) were compared with the anchor-rater's scoring. LLMs were provided with both single-shot and few-shot prompting strategies to optimize performance. RESULTS: Overall assessment accuracy was significantly higher among GI fellows (72.4 %) compared to specialists (65.3 %, p = 0.004) and LLMs (58.9 %, p < 0.001). In the detection of edema, LLMs outperformed specialists (83.3 % vs 49.3 %, p < 0.001). However, LLMs showed significantly poor performance in rings assessment (30 %) compared to specialists (58 %) and fellows (58.7 %, both p < 0.001). After implementation of few-shot prompting, the overall performance of LLMs was comparable to GI specialists (62.7 % vs 65.3 %, p = 0.3). CONCLUSIONS: This study uncovers variability in EREFS scoring across human raters with different expertise and multimodal LLMs, and demonstrates that few-shot prompting can optimize LLM accuracy.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Eosinophilic EsophagitisGastroenterologistsGastroenterologyClinical CompetenceEsophagoscopyHumansFemaleMaleAdultSeverity of Illness IndexMiddle Aged

Stichwörter der Autor:innen

EREFSEndoscopyEosinophilic EsophagitisLarge-language models (LLMs)Prompt engineering

Autor:innen und Institutionen

  • Department of Gastroenterology, Sheba Medical Center, Israel; Gray Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel. Electronic address: alavra@gmail.com.
  • Department of Gastroenterology, Sheba Medical Center, Israel; Gray Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
  • Department of Surgery, Oncology and Gastroenterology, University of Padua, Padua, Italy; Gastroenterology Unit, University Hospital of Padua, Padua, Italy.

Zitieren

APA

Levartovsky, A., Dar, L., Fay, S., Ukashi, O., Shachar, E., Engel, T., Ben-Horin, S., Savarino, E. V., & Ungar, B. (2025). Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis. Digestive and Liver Disease. https://doi.org/10.1016/j.dld.2025.11.009

Vancouver

Levartovsky A, Dar L, Fay S, Ukashi O, Shachar E, Engel T, et al. Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis. Digestive and Liver Disease. 2025. doi:10.1016/j.dld.2025.11.009

BibTeX
@article{Levartovsky2025,
  author = {Levartovsky, Asaf and Dar, Lior and Fay, Shmuel and Ukashi, Offir and Shachar, Eyal and Engel, Tal and Ben-Horin, Shomron and Savarino, Edoardo V and Ungar, Bella},
  title = {Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis},
  journal = {Digestive and Liver Disease},
  year = {2025},
  doi = {10.1016/j.dld.2025.11.009}
}
Kostenlos starten →