Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis
Digestive and Liver Disease · 2025
DOI: 10.1016/j.dld.2025.11.009 ↗Kurz gefasst
Fachartikel in der Zeitschrift Digestive and Liver Disease, 2025. Verfasst von Asaf Levartovsky und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Gastroenterology, Sheba Medical Center, Israel; Gray Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel. Electronic address: al…. Hauptthemen laut MEDLINE: Eosinophilic Esophagitis, Gastroenterologists, Gastroenterology, Clinical Competence. Die Arbeit nennt 31 Literaturangaben.
Abstract (Originalsprache)
BACKGROUND: The comprehensive evaluation of multimodal large language models (LLMs) in gastroenterological image-analysis remains limited. We aimed to assess the accuracy of the Eosinophilic Esophagitis Endoscopic Reference Score (EREFS) scoring system for Eosinophilic Esophagitis (EoE) across gastroenterology (GI) clinicians with varying levels of experience. METHODS: Fifty real-world endoscopic images of EoE patients were graded based on the original EREFS score by a gold standard anchor-rater. EREFS gradings of GI specialists, fellows and three multimodal LLMs (ChatGPT-4o, Claude Sonnet 3.5, Perplexity Sonar) were compared with the anchor-rater's scoring. LLMs were provided with both single-shot and few-shot prompting strategies to optimize performance. RESULTS: Overall assessment accuracy was significantly higher among GI fellows (72.4 %) compared to specialists (65.3 %, p = 0.004) and LLMs (58.9 %, p < 0.001). In the detection of edema, LLMs outperformed specialists (83.3 % vs 49.3 %, p < 0.001). However, LLMs showed significantly poor performance in rings assessment (30 %) compared to specialists (58 %) and fellows (58.7 %, both p < 0.001). After implementation of few-shot prompting, the overall performance of LLMs was comparable to GI specialists (62.7 % vs 65.3 %, p = 0.3). CONCLUSIONS: This study uncovers variability in EREFS scoring across human raters with different expertise and multimodal LLMs, and demonstrates that few-shot prompting can optimize LLM accuracy.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Gastroenterology, Sheba Medical Center, Israel; Gray Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel. Electronic address: alavra@gmail.com.
- Department of Gastroenterology, Sheba Medical Center, Israel; Gray Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
- Department of Surgery, Oncology and Gastroenterology, University of Padua, Padua, Italy; Gastroenterology Unit, University Hospital of Padua, Padua, Italy.
Zitieren
APA
Levartovsky, A., Dar, L., Fay, S., Ukashi, O., Shachar, E., Engel, T., Ben-Horin, S., Savarino, E. V., & Ungar, B. (2025). Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis. Digestive and Liver Disease. https://doi.org/10.1016/j.dld.2025.11.009
Vancouver
Levartovsky A, Dar L, Fay S, Ukashi O, Shachar E, Engel T, et al. Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis. Digestive and Liver Disease. 2025. doi:10.1016/j.dld.2025.11.009
BibTeX
@article{Levartovsky2025,
author = {Levartovsky, Asaf and Dar, Lior and Fay, Shmuel and Ukashi, Offir and Shachar, Eyal and Engel, Tal and Ben-Horin, Shomron and Savarino, Edoardo V and Ungar, Bella},
title = {Performance of gastroenterologists and multimodal LLMs in endoscopic EREFS scoring of Eosinophilic Esophagitis},
journal = {Digestive and Liver Disease},
year = {2025},
doi = {10.1016/j.dld.2025.11.009}
}
EUVIMED