Large Language Model‐Driven Analysis and Report Generation of Endoscopy Videos—A Pilot Study
Digestive Endoscopy · 2026-03 · 38 (3)
DOI: 10.1111/den.70134 ↗Kurz gefasst
Fachartikel in der Zeitschrift Digestive Endoscopy, online erschienen am 10.03.2026. Verfasst von Davide Massimi und 16 weiteren Autor:innen. Beteiligte Einrichtung: IRCCS Humanitas Research Hospital, Milan, Italy. Hauptthemen laut MEDLINE: Video Recording, Endoscopy, Digestive System. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 6 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fondazione AIRC per la ricerca sul cancro ETS, European Commission (Horizon Europe), Norwegian Research Council.
Abstract (Originalsprache)
Multimodal large language models (MLLMs) can automatically analyze clinical video, but evidence from full esophagogastroduodenoscopy (EGD) and the impact of on‐screen computer‐aided detection/diagnosis (CAD) overlays on MLLM behavior remain unclear. We tested whether an MLLM can produce clinically adequate EGD reports and whether a CAD overlay changes performance. We analyzed five complete EGD videos with Gemini 2.5 Pro in paired versions: (1) clean video and (2) the same video with a CAD overlay. Five blinded endoscopists rated report adequacy in three domains. MLLM accuracy for landmarks/lesions was further assessed by two blinded expert endoscopists using the time‐window rule (a model detection counted as correct if it occurred within ±2 s of the expert‐annotated timestamp). In this retrospective pilot study, five archived diagnostic EGD procedures from five patients were available as full‐length videos. Across five raters, MLLM Completeness was judged adequate in 56.0% (14/25 ratings) with Clean‐Video versus 48.0% (12/25 ratings) with Overlay‐Video ( p = 0.500). Visualization was identical (36.0% [9/25 ratings] for both; p = 1.000). Lesions characteristics were identical (16.0% [4/25] for both; p = 1.00). For the Landmark agreement, the overall accuracy of the MLLM with Clean‐Video vs. Overlay‐Video was: 0.55 [95% CI 0.43–0.67] vs. 0.33 [0.23–0.46], p = 0.029; sensitivity 0.53 [0.40–0.66] vs. 0.35 [0.24–0.49], p = 0.122; specificity 0.67 [0.35–0.88] vs. 0.22 [0.06–0.55], p = 0.125. In this pilot study, Gemini 2.5 Pro demonstrated inadequate performance for clinical EGD reporting. These hypothesis‐generating findings suggest substantial optimization and larger‐scale validation are required before deployment.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- IRCCS Humanitas Research Hospital, Milan, Italy.
- Clinical Effectiveness Research Group, University of Oslo, Oslo, Norway.
- Digestive Disease Center, Showa University Northern Yokohama Hospital, Yokohama, Japan.
- Gastroenterology and Digestive Endoscopy Unit, Ospedale dei Castelli, Rome, Italy.
- Division of Gastroenterology, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada.
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, TARGID, KU Leuven, Leuven, Belgium.
ORCID-Profile: Tommy Rizkala, Kareem Khalaf, Raf Bisschops, Cesare Hassan
Förderung
- Fondazione AIRC per la ricerca sul cancro ETS
- European Commission (Horizon Europe)
- Norwegian Research Council
- Flanders
- KU Leuven
- HORIZON EUROPE Framework Programme
- Norges Forskningsråd
- Vlaamse regering
Zitieren
APA
Massimi, D., Stefano, L. D., Rizkala, T., Spadaccini, M., Mori, Y., Menini, M., Antonelli, G., Khalaf, K., Bisschops, R., von Renteln, D., Sharma, P., Rex, D. K., Bretthauer, M., Castoro, C., LLM Working Group, Repici, A., & Hassan, C. (2026). Large Language Model‐Driven Analysis and Report Generation of Endoscopy Videos—A Pilot Study. Digestive Endoscopy, 38(3). https://doi.org/10.1111/den.70134
Vancouver
Massimi D, Stefano LD, Rizkala T, Spadaccini M, Mori Y, Menini M, et al. Large Language Model‐Driven Analysis and Report Generation of Endoscopy Videos—A Pilot Study. Digestive Endoscopy. 2026;38(3). doi:10.1111/den.70134
BibTeX
@article{Massimi2026,
author = {Massimi, Davide and Stefano, Luca Di and Rizkala, Tommy and Spadaccini, Marco and Mori, Yuichi and Menini, Maddalena and Antonelli, Giulio and Khalaf, Kareem and Bisschops, Raf and von Renteln, Daniel and Sharma, Prateek and Rex, Douglas K and Bretthauer, Michael and Castoro, Carlo and LLM Working Group, LLM Working Group and Repici, Alessandro and Hassan, Cesare},
title = {Large Language Model‐Driven Analysis and Report Generation of Endoscopy Videos—A Pilot Study},
journal = {Digestive Endoscopy},
year = {2026},
volume = {38},
number = {3},
doi = {10.1111/den.70134}
}
EUVIMED