Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning
Surgical Endoscopy · 2025-03 · 39 (3) · 1709-1719
DOI: 10.1007/s00464-024-11486-3 ↗Kurz gefasst
Fachartikel in der Zeitschrift Surgical Endoscopy, online erschienen am 13.01.2025. Verfasst von Nasseh Hashemi und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Clinical Medicine, Aalborg University, Aalborg, Denmark. nasseh.hashemi@gmail.com. Hauptthemen laut MEDLINE: Deep Learning, Robotic Surgical Procedures, Clinical Competence. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Klinisk Institut, Aalborg Universitet, Aalborg University.
Abstract (Originalsprache)
Objectives
This study aimed to develop an automated skills assessment tool for surgical trainees using deep learning.
Background
Optimal surgical performance in robot-assisted surgery (RAS) is essential for ensuring good surgical outcomes. This requires effective training of new surgeons, which currently relies on supervision and skill assessment by experienced surgeons. Artificial Intelligence (AI) presents an opportunity to augment existing human-based assessments.
Methods
We used a network architecture consisting of a convolutional neural network combined with a long short-term memory (LSTM) layer to create two networks for the extraction and analysis of spatial and temporal features from video recordings of surgical procedures, facilitating action recognition and skill assessment.
Results
21 participants (16 novices and 5 experienced) performed 16 different intra-abdominal robot-assisted surgical procedures on porcine models. The action recognition network achieved an accuracy of 96.0% in identifying surgical actions. A GradCAM filter was used to enhance the model interpretability. The skill assessment network had an accuracy of 81.3% in classifying novices and experiences. Procedure plots were created to visualize the skill assessment.
Conclusion
Our study demonstrated that AI can be used to automate surgical action recognition and skill assessment. The use of a porcine model enables effective data collection at different levels of surgical performance, which is normally not available in the clinical setting. Future studies need to test how well AI developed within a porcine setting can be used to detect errors and provide feedback and actionable skills assessment in the clinical setting.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Clinical Medicine, Aalborg University, Aalborg, Denmark. nasseh.hashemi@gmail.com.
- Nordsim - Centre for Skills Training and Simulation, Aalborg University Hospital, Aalborg, Denmark. nasseh.hashemi@gmail.com.
- ROCnord - Robot Centre, Aalborg University Hospital, Aalborg, Denmark. nasseh.hashemi@gmail.com.
- Department of Urology, Aalborg University Hospital, Aalborg, Denmark. nasseh.hashemi@gmail.com.
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
- Copenhagen Academy for Medical Education and Simulation, Center for Human Resources and Education, The Capital Region of Denmark, Copenhagen, Denmark.
ORCID-Profile: Nasseh Hashemi
Förderung
- Klinisk Institut, Aalborg Universitet
- Aalborg University
Zitieren
APA
Hashemi, N., Mose, M., Østergaard, L. R., Bjerrum, F., Hashemi, M., Svendsen, M. B. S., Friis, M. L., Tolsgaard, M. G., & Rasmussen, S. (2025). Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning. Surgical Endoscopy, 39(3), 1709-1719. https://doi.org/10.1007/s00464-024-11486-3
Vancouver
Hashemi N, Mose M, Østergaard LR, Bjerrum F, Hashemi M, Svendsen MBS, et al. Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning. Surgical Endoscopy. 2025;39(3):1709-1719. doi:10.1007/s00464-024-11486-3
BibTeX
@article{Hashemi2025,
author = {Hashemi, Nasseh and Mose, Matias and Østergaard, Lasse R and Bjerrum, Flemming and Hashemi, Mostaan and Svendsen, Morten B S and Friis, Mikkel L and Tolsgaard, Martin G and Rasmussen, Sten},
title = {Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning},
journal = {Surgical Endoscopy},
year = {2025},
volume = {39},
number = {3},
pages = {1709--1719},
doi = {10.1007/s00464-024-11486-3}
}
EUVIMED