Intraprocedural instrument recognition during endoscopic submucosal dissection using artificial intelligence
Endoscopy · 01.06.2026 · 58 (S 03 S 03) · S708-S709
DOI: 10.1055/s-0046-1822020 ↗Kurz gefasst
Fachartikel in der Zeitschrift Endoscopy, erschienen am 01.06.2026. Verfasst von M W Scheppach und 8 weiteren Autor:innen.
Abstract (Originalsprache)
Aims
The automation of procedure analysis for endoscopic interventions is a prerequisite for the advancement towards automatic reporting, quality assessment and advanced operator assistence systems. A critical aspect of procedure characteristics is the choice and utilization of endoscopic instruments. The endoscopic submucosal dissection (ESD) is a complex resection technique with a slow learning curve and relevant operator dependent risks for procedure related complications such as bleeding and perforation. During ESD, specific instruments such as electrosurgical knives of different shapes or hemostatic forceps are used. The analysis of the utilized endoscopic instruments may facilitate a more profound understanding of procedure characteristics. Therefore, we aimed to develop an algorithm for the automatic detection and delineation of endoscopic instruments during ESD ([ Table 1 ]). Table 1 Class Sensitivity Specificity Dice Background 0.98 0.91 0.97 Hook knife Shaft 0.92 1.00 0.93 Hook knife Tip 0.78 1.00 0.76 Dual knife Shaft 0.93 1.00 0.95 Dual knife Tip 0.46 1.00 0.54 Flush knife Shaft 0.93 1.00 0.92 Flush knife Tip 0.08 1.00 0.14 Needle Shaft 0.96 1.00 0.91 Needle Tip 0.82 1.00 0.79 Hemostatic forceps Shaft 0.30 1.00 0.41 Hemostatic forceps Tip 0.57 1.00 0.52 Clip Shaft 0.84 1.00 0.84 Clip Tip 0.31 1.00 0.44 Distal Attachment 0.90 0.98 0.93 Methods 65 full length ESD videos (15x rectal, 29x esophageal, 16x gastric, 5x duodenal ESD) were extracted from Augsburg University Hospital database. 8.165 single images from these procedures were used for training. On these images, shaft and tip of endoscopic instruments were delineated by two study investigators using Computer Vision and Annotation Tool. Classes included Hook knife, Dual knife (both Olympus, Tokyo, Japan), Flush knife (Fujifilm, Tokyo, Japan), injection needle, hemostatic forceps, hemostatic clip and transparent distal attachment. Annotated images were used for training of a DeepLabV3+deep learning algorithm. 5-fold internal cross validation was performed on pixel basis, as well as for instrument detection per image (sensitivity, specificity, F1 score). For the instrument detection, a threshhold of 0.2 for the intersection over union of algorithm prediction and ground truth was used.
Results
Internal validation results on pixel basis are shown in the table. For the detection task, F1 scores of 1.00, 0.90, 0.88, 0.65, 0.45, 0.68 and 0.99 were measured for Hook knife, Dual knife, Flush knife, needle, hemostatic forceps, hemostatic clip and distal attachment.
Conclusions
In this exploratory study, endoscopic instruments were detected and delineated with varying performance by a deep learning algorithm. This was in part attributed to class imbalances in the training data. However, the incorporation of temporal information may also improve performance. Further research should focus on evaluating this new technology for automatic reporting of ESD procedures. Publication History Article published online: 05 June 2026 © 2026. European Society of Gastrointestinal Endoscopy. All rights reserved. Georg Thieme Verlag KG Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany
Zitieren
APA
Scheppach, M. W., Rauber, D., Büchele, C., Nunes, D. W., Nagl, S., Probst, A., Römmele, C., Palm, C., & Messmann, H. (2026). Intraprocedural instrument recognition during endoscopic submucosal dissection using artificial intelligence. Endoscopy, 58(S 03 S 03), S708-S709. https://doi.org/10.1055/s-0046-1822020
Vancouver
Scheppach MW, Rauber D, Büchele C, Nunes DW, Nagl S, Probst A, et al. Intraprocedural instrument recognition during endoscopic submucosal dissection using artificial intelligence. Endoscopy. 2026;58(S 03 S 03):S708-S709. doi:10.1055/s-0046-1822020
BibTeX
@article{Scheppach2026,
author = {Scheppach, M W and Rauber, D and Büchele, C and Nunes, D Weber and Nagl, S and Probst, A and Römmele, C and Palm, C and Messmann, H},
title = {Intraprocedural instrument recognition during endoscopic submucosal dissection using artificial intelligence},
journal = {Endoscopy},
year = {2026},
volume = {58},
number = {S 03 S 03},
pages = {S708--S709},
doi = {10.1055/s-0046-1822020}
}
EUVIMED