Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition
IEEE Transactions on Medical Imaging · 2023-09 · 42 (9) · 2592-2602
DOI: 10.1109/tmi.2023.3262847 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2023-09. Verfasst von Sanat Ramesh und 8 weiteren Autor:innen. Hauptthemen laut MEDLINE: Neural Networks, Computer, Surgery, Computer-Assisted. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union’s Horizon 2020 Research and Innovation Program, Marie Sklodowska-Curie, AuTonomous intraLuminAl Surgery.
Abstract (Originalsprache)
Automatic recognition of fine-grained surgical activities, called steps, is a challenging but crucial task for intelligent intra-operative computer assistance. The development of current vision-based activity recognition methods relies heavily on a high volume of manually annotated data. This data is difficult and time-consuming to generate and requires domain-specific knowledge. In this work, we propose to use coarser and easier-to-annotate activity labels, namely phases, as weak supervision to learn step recognition with fewer step annotated videos. We introduce a step-phase dependency loss to exploit the weak supervision signal. We then employ a Single-Stage Temporal Convolutional Network (SS-TCN) with a ResNet-50 backbone, trained in an end-to-end fashion from weakly annotated videos, for temporal activity segmentation and recognition. We extensively evaluate and show the effectiveness of the proposed method on a large video dataset consisting of 40 laparoscopic gastric bypass procedures and the public benchmark CATARACTS containing 50 cataract surgeries.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- European Union’s Horizon 2020 Research and Innovation Program
- Marie Sklodowska-Curie
- AuTonomous intraLuminAl Surgery
- French State Funds managed within the Investissements d’Avenir Program
- Banque Publique d’Investissement (BPI) France Project Connected Optimized Network and Data in Operating Rooms
- ANR
Zitieren
APA
Ramesh, S., Dall'Alba, D., Gonzalez, C., Yu, T., Mascagni, P., Mutter, D., Marescaux, J., Fiorini, P., & Padoy, N. (2023). Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition. IEEE Transactions on Medical Imaging, 42(9), 2592-2602. https://doi.org/10.1109/tmi.2023.3262847
Vancouver
Ramesh S, Dall'Alba D, Gonzalez C, Yu T, Mascagni P, Mutter D, et al. Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition. IEEE Transactions on Medical Imaging. 2023;42(9):2592-2602. doi:10.1109/tmi.2023.3262847
BibTeX
@article{Ramesh2023,
author = {Ramesh, Sanat and Dall'Alba, Diego and Gonzalez, Cristians and Yu, Tong and Mascagni, Pietro and Mutter, Didier and Marescaux, Jacques and Fiorini, Paolo and Padoy, Nicolas},
title = {Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition},
journal = {IEEE Transactions on Medical Imaging},
year = {2023},
volume = {42},
number = {9},
pages = {2592--2602},
doi = {10.1109/tmi.2023.3262847}
}
EUVIMED