ChatGPT4 Vision – Can it do endoscopy?
Endoscopy · 01.04.2024 · 56 (S 02 S 02) · S384-S385
DOI: 10.1055/s-0044-1783649 ↗Kurz gefasst
Fachartikel in der Zeitschrift Endoscopy, erschienen am 01.04.2024. Verfasst von D. Schulz und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Aims
ChatGPT4 is the one of the most successful and most wide known large language artificial intelligence models (LLM) available today. Until September 2023, it only accepted text prompts. Now, it has the ability to interpret pictures and give a text description of them. In this work, we tested the model on various upper and lower endoscopic images.
Methods
We uploaded 10 anonymized images each of anatomical sites without any visible pathology including esophagus, stomach, pylorus, duodenum, ileum and colon, respectively, and asked ChatGPT4 “What do you see?”. Furthermore, we uploaded 10 endoscopic images each of pathologies, including varices ( 8 gastric and 2 fundus), gastric ulcers, colon polyps, colon cancer and colonic diverticula, respectively, and again used “What do you see?” as the prompt. We rated first whether the model recognized that it was a picture from an endoscopy and secondly, whether it could identify the correct anatomical site or the pathology.
Results
Chat-GPT4 Vision recognized 109 of 110 images as endoscopic images. For the anatomical sites, the model correctly identified esophagus 7/10, stomach 1/10, pylorus 2/10, duodenum 0/10, ileum 1/10, and colon 1/10. For the pathologies, the model identified correctly 0/10 varices, 3/10 gastric ulcers, 9/10 colon polyps, 1/10 colon cancer and 1/10 colonic diverticula.
Conclusions
While Chat-GPT4 Vision showed some impressive results in colonoscopy images in this small study and could readily identify 90% of the pictures with polyps, it currently lacks the ability to interpret GI endoscopic sites and other pathologies. LLM may need domain-specific training to solve endoscopy-specific tasks. Publication History Article published online: 15 April 2024 © 2024. European Society of Gastrointestinal Endoscopy. All rights reserved. Georg Thieme Verlag KG Rüdigerstraße 14, 70469 Stuttgart, Germany
Zitieren
APA
Schulz, D., Ebigbo, A., & Messmann, H. (2024). ChatGPT4 Vision – Can it do endoscopy?. Endoscopy, 56(S 02 S 02), S384-S385. https://doi.org/10.1055/s-0044-1783649
Vancouver
Schulz D, Ebigbo A, Messmann H. ChatGPT4 Vision – Can it do endoscopy?. Endoscopy. 2024;56(S 02 S 02):S384-S385. doi:10.1055/s-0044-1783649
BibTeX
@article{Schulz2024,
author = {Schulz, D. and Ebigbo, A. and Messmann, H.},
title = {ChatGPT4 Vision – Can it do endoscopy?},
journal = {Endoscopy},
year = {2024},
volume = {56},
number = {S 02 S 02},
pages = {S384--S385},
doi = {10.1055/s-0044-1783649}
}
EUVIMED