EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning

Nasseh Hashemi iD, Matias Mose, Lasse R Østergaard, Flemming Bjerrum, Mostaan Hashemi, Morten B S Svendsen, Mikkel L Friis, Martin G Tolsgaard, Sten Rasmussen

Surgical Endoscopy · 2025-03 · 39 (3) · 1709-1719

DOI: 10.1007/s00464-024-11486-3 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Surgical Endoscopy, online erschienen am 13.01.2025. Verfasst von Nasseh Hashemi und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Clinical Medicine, Aalborg University, Aalborg, Denmark. nasseh.hashemi@gmail.com. Hauptthemen laut MEDLINE: Deep Learning, Robotic Surgical Procedures, Clinical Competence. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Klinisk Institut, Aalborg Universitet, Aalborg University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

This study aimed to develop an automated skills assessment tool for surgical trainees using deep learning.

Background

Optimal surgical performance in robot-assisted surgery (RAS) is essential for ensuring good surgical outcomes. This requires effective training of new surgeons, which currently relies on supervision and skill assessment by experienced surgeons. Artificial Intelligence (AI) presents an opportunity to augment existing human-based assessments.

Methods

We used a network architecture consisting of a convolutional neural network combined with a long short-term memory (LSTM) layer to create two networks for the extraction and analysis of spatial and temporal features from video recordings of surgical procedures, facilitating action recognition and skill assessment.

Results

21 participants (16 novices and 5 experienced) performed 16 different intra-abdominal robot-assisted surgical procedures on porcine models. The action recognition network achieved an accuracy of 96.0% in identifying surgical actions. A GradCAM filter was used to enhance the model interpretability. The skill assessment network had an accuracy of 81.3% in classifying novices and experiences. Procedure plots were created to visualize the skill assessment.

Conclusion

Our study demonstrated that AI can be used to automate surgical action recognition and skill assessment. The use of a porcine model enables effective data collection at different levels of surgical performance, which is normally not available in the clinical setting. Future studies need to test how well AI developed within a porcine setting can be used to detect errors and provide feedback and actionable skills assessment in the clinical setting.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Deep LearningRobotic Surgical ProceduresClinical CompetenceAnimalsSwineVideo RecordingModels, AnimalHumansNeural Networks, Computer

Stichwörter der Autor:innen

Action regcognitionArtificial intelligenceRobot-assisted surgerySkills assessment

Autor:innen und Institutionen

  • Department of Clinical Medicine, Aalborg University, Aalborg, Denmark. nasseh.hashemi@gmail.com.
  • Nordsim - Centre for Skills Training and Simulation, Aalborg University Hospital, Aalborg, Denmark. nasseh.hashemi@gmail.com.
  • ROCnord - Robot Centre, Aalborg University Hospital, Aalborg, Denmark. nasseh.hashemi@gmail.com.
  • Department of Urology, Aalborg University Hospital, Aalborg, Denmark. nasseh.hashemi@gmail.com.
  • Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
  • Copenhagen Academy for Medical Education and Simulation, Center for Human Resources and Education, The Capital Region of Denmark, Copenhagen, Denmark.

ORCID-Profile: Nasseh Hashemi

Förderung

  • Klinisk Institut, Aalborg Universitet
  • Aalborg University

Zitieren

APA

Hashemi, N., Mose, M., Østergaard, L. R., Bjerrum, F., Hashemi, M., Svendsen, M. B. S., Friis, M. L., Tolsgaard, M. G., & Rasmussen, S. (2025). Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning. Surgical Endoscopy, 39(3), 1709-1719. https://doi.org/10.1007/s00464-024-11486-3

Vancouver

Hashemi N, Mose M, Østergaard LR, Bjerrum F, Hashemi M, Svendsen MBS, et al. Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning. Surgical Endoscopy. 2025;39(3):1709-1719. doi:10.1007/s00464-024-11486-3

BibTeX
@article{Hashemi2025,
  author = {Hashemi, Nasseh and Mose, Matias and Østergaard, Lasse R and Bjerrum, Flemming and Hashemi, Mostaan and Svendsen, Morten B S and Friis, Mikkel L and Tolsgaard, Martin G and Rasmussen, Sten},
  title = {Video-based robotic surgical action recognition and skills assessment on porcine models using deep learning},
  journal = {Surgical Endoscopy},
  year = {2025},
  volume = {39},
  number = {3},
  pages = {1709--1719},
  doi = {10.1007/s00464-024-11486-3}
}
Kostenlos starten →