EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition

Sanat Ramesh, Diego Dall'Alba, Cristians Gonzalez, Tong Yu, Pietro Mascagni, Didier Mutter, Jacques Marescaux, Paolo Fiorini, Nicolas Padoy

IEEE Transactions on Medical Imaging · 2023-09 · 42 (9) · 2592-2602

DOI: 10.1109/tmi.2023.3262847 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2023-09. Verfasst von Sanat Ramesh und 8 weiteren Autor:innen. Hauptthemen laut MEDLINE: Neural Networks, Computer, Surgery, Computer-Assisted. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union’s Horizon 2020 Research and Innovation Program, Marie Sklodowska-Curie, AuTonomous intraLuminAl Surgery.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automatic recognition of fine-grained surgical activities, called steps, is a challenging but crucial task for intelligent intra-operative computer assistance. The development of current vision-based activity recognition methods relies heavily on a high volume of manually annotated data. This data is difficult and time-consuming to generate and requires domain-specific knowledge. In this work, we propose to use coarser and easier-to-annotate activity labels, namely phases, as weak supervision to learn step recognition with fewer step annotated videos. We introduce a step-phase dependency loss to exploit the weak supervision signal. We then employ a Single-Stage Temporal Convolutional Network (SS-TCN) with a ResNet-50 backbone, trained in an end-to-end fashion from weakly annotated videos, for temporal activity segmentation and recognition. We extensively evaluate and show the effectiveness of the proposed method on a large video dataset consisting of 40 laparoscopic gastric bypass procedures and the public benchmark CATARACTS containing 50 cataract surgeries.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Neural Networks, ComputerSurgery, Computer-Assisted

Förderung

  • European Union’s Horizon 2020 Research and Innovation Program
  • Marie Sklodowska-Curie
  • AuTonomous intraLuminAl Surgery
  • French State Funds managed within the Investissements d’Avenir Program
  • Banque Publique d’Investissement (BPI) France Project Connected Optimized Network and Data in Operating Rooms
  • ANR

Zitieren

APA

Ramesh, S., Dall'Alba, D., Gonzalez, C., Yu, T., Mascagni, P., Mutter, D., Marescaux, J., Fiorini, P., & Padoy, N. (2023). Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition. IEEE Transactions on Medical Imaging, 42(9), 2592-2602. https://doi.org/10.1109/tmi.2023.3262847

Vancouver

Ramesh S, Dall'Alba D, Gonzalez C, Yu T, Mascagni P, Mutter D, et al. Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition. IEEE Transactions on Medical Imaging. 2023;42(9):2592-2602. doi:10.1109/tmi.2023.3262847

BibTeX
@article{Ramesh2023,
  author = {Ramesh, Sanat and Dall'Alba, Diego and Gonzalez, Cristians and Yu, Tong and Mascagni, Pietro and Mutter, Didier and Marescaux, Jacques and Fiorini, Paolo and Padoy, Nicolas},
  title = {Weakly Supervised Temporal Convolutional Networks for Fine-Grained Surgical Activity Recognition},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2023},
  volume = {42},
  number = {9},
  pages = {2592--2602},
  doi = {10.1109/tmi.2023.3262847}
}
Kostenlos starten →