PolypNextLSTM: a lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM
International Journal of Computer Assisted Radiology and Surgery · 08.08.2024 · 19 (10) · 2111-2119
DOI: 10.1007/s11548-024-03244-6 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, online erschienen am 08.08.2024. Verfasst von Debayan Bhattacharya und 5 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Medical Technology and Intelligent Systems, Technische Universitaet Hamburg, Hamburg, Germany. Hauptthemen laut MEDLINE: Deep Learning, Video Recording. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Technische Universität Hamburg.
Abstract (Originalsprache)
Purpose
Commonly employed in polyp segmentation, single-image UNet architectures lack the temporal insight clinicians gain from video data in diagnosing polyps. To mirror clinical practices more faithfully, our proposed solution, PolypNextLSTM , leverages video-based deep learning, harnessing temporal information for superior segmentation performance with least parameter overhead, making it possibly suitable for edge devices.
Methods
PolypNextLSTM employs a UNet-like structure with ConvNext-Tiny as its backbone, strategically omitting the last two layers to reduce parameter overhead. Our temporal fusion module, a Convolutional Long Short Term Memory (ConvLSTM), effectively exploits temporal features. Our primary novelty lies in PolypNextLSTM , which stands out as the leanest in parameters and the fastest model, surpassing the performance of five state-of-the-art image and video-based deep learning models. The evaluation of the SUN-SEG dataset spans easy-to-detect and hard-to-detect polyp scenarios, along with videos containing challenging artefacts like fast motion and occlusion.
Results
Comparison against 5 image-based and 5 video-based models demonstrates PolypNextLSTM ’s superiority, achieving a Dice score of 0.7898 on the hard-to-detect polyp test set, surpassing image-based PraNet (0.7519) and video-based PNS+ (0.7486). Notably, our model excels in videos featuring complex artefacts such as ghosting and occlusion.
Conclusion
PolypNextLSTM , integrating pruned ConvNext-Tiny with ConvLSTM for temporal fusion, not only exhibits superior segmentation performance but also maintains the highest frames per speed among evaluated models. Code can be found here: https://github.com/mtec-tuhh/PolypNextLSTM .
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute of Medical Technology and Intelligent Systems, Technische Universitaet Hamburg, Hamburg, Germany.
- Institute of Medical Technology and Intelligent Systems, Technische Universitaet Hamburg, Hamburg, Germany. konrad.reuter@tuhh.de.
Förderung
- Technische Universität Hamburg
Zitieren
APA
Bhattacharya, D., Reuter, K., Behrendt, F., Maack, L., Grube, S., & Schlaefer, A. (2024). PolypNextLSTM: a lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM. International Journal of Computer Assisted Radiology and Surgery, 19(10), 2111-2119. https://doi.org/10.1007/s11548-024-03244-6
Vancouver
Bhattacharya D, Reuter K, Behrendt F, Maack L, Grube S, Schlaefer A. PolypNextLSTM: a lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM. International Journal of Computer Assisted Radiology and Surgery. 2024;19(10):2111-2119. doi:10.1007/s11548-024-03244-6
BibTeX
@article{Bhattacharya2024,
author = {Bhattacharya, Debayan and Reuter, Konrad and Behrendt, Finn and Maack, Lennart and Grube, Sarah and Schlaefer, Alexander},
title = {PolypNextLSTM: a lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2024},
volume = {19},
number = {10},
pages = {2111--2119},
doi = {10.1007/s11548-024-03244-6}
}
EUVIMED