eNCApsulate: neural cellular automata for precision diagnosis on capsule endoscopes
International Journal of Computer Assisted Radiology and Surgery · 2025
DOI: 10.1007/s11548-025-03425-x ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, 2025. Verfasst von Henry John Krumb und 1 weiteren Autor:innen. Beteiligte Einrichtung: Computer Science Department, TU Darmstadt, Fraunhoferstr. 5, Darmstadt, 64293, Hessen, Germany. henry_john.krumb@tu-darmstadt.de. Hauptthemen laut MEDLINE: Capsule Endoscopy, Neural Networks, Computer, Capsule Endoscopes, Image Processing, Computer-Assisted. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Norges Forskningsråd.
Abstract (Originalsprache)
Purpose
Wireless capsule endoscopy (WCE) is a noninvasive imaging method for the entire gastrointestinal tract and is a pain-free alternative to traditional endoscopy. It generates extensive video data that requires significant review time, and localizing the capsule after ingestion is a challenge. Techniques like bleeding detection and depth estimation can help with localization of pathologies, but deep learning models are typically too large to run directly on the capsule.
Methods
Neural cellular automata (NCAs) for bleeding segmentation and depth estimation are trained on capsule endoscopic images. For monocular depth estimation, we distill a large foundation model into the lean NCA architecture, by treating the outputs of the foundation model as pseudo-ground truth. We then port the trained NCAs to the ESP32 microcontroller, enabling efficient image processing on hardware as small as a camera capsule.
Results
NCAs are more accurate (Dice) than other portable segmentation models, while requiring more than 100x fewer parameters stored in memory than other small-scale models. The visual results of NCAs depth estimation look convincing and in some cases beat the realism and detail of the pseudo-ground truth. Runtime optimizations on the ESP32-S3 accelerate the average inference speed significantly, by more than factor 3.
Conclusion
With several algorithmic adjustments and distillation, it is possible to eNCApsulate NCA models into microcontrollers that fit into wireless capsule endoscopes. This is the first work that enables reliable bleeding segmentation and depth estimation on a miniaturized device, paving the way for precise diagnosis combined with visual odometry as a means of precise localization of the capsule—on the capsule.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Computer Science Department, TU Darmstadt, Fraunhoferstr. 5, Darmstadt, 64293, Hessen, Germany. henry_john.krumb@tu-darmstadt.de.
- Computer Science Department, TU Darmstadt, Fraunhoferstr. 5, Darmstadt, 64293, Hessen, Germany.
Förderung
- Norges Forskningsråd
Zitieren
APA
Krumb, H. J., & Mukhopadhyay, A. (2025). eNCApsulate: neural cellular automata for precision diagnosis on capsule endoscopes. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-025-03425-x
Vancouver
Krumb HJ, Mukhopadhyay A. eNCApsulate: neural cellular automata for precision diagnosis on capsule endoscopes. International Journal of Computer Assisted Radiology and Surgery. 2025. doi:10.1007/s11548-025-03425-x
BibTeX
@article{Krumb2025,
author = {Krumb, Henry John and Mukhopadhyay, Anirban},
title = {eNCApsulate: neural cellular automata for precision diagnosis on capsule endoscopes},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2025},
doi = {10.1007/s11548-025-03425-x}
}
EUVIMED