EUVIMEDEUVIMED entdecken
Crossref

Deep-Learning Enhanced Augmented Reality for Robotic Radical Prostatectomy on Consumer Grade Hardware

Vincent Benard iD, Olivier Windisch iD, Mina Bjelogrlic, Hugues Turbé, Christian Lovis, Michele Diana, Massimo Valerio

Surgical Innovation · 06.11.2026

DOI: 10.1177/15533506261504154 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Surgical Innovation, online erschienen am 06.11.2026. Verfasst von Vincent Benard und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Urology, University Hospitals of Geneva, Geneva, Switzerland. Die Arbeit nennt 47 Literaturangaben. Gefördert von: Foundation Nicod.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Augmented Reality (AR) during Robotic-Assisted Radical Prostatectomy (RARP) provides surgical guidance by overlaying patient-specific 3D Virtual Models (VM) onto the intraoperative view. A main drawback is instrument occlusion, where VM overlaps the surgical tools, disturbing the augmented view. Deep learning (DL)-based desocclusion addresses this issue, but current implementation rely on expensive, specialized hardware.

Methods

We developed an AR desocclusion pipeline running on consumer-grade hardware (∼1150 US Dollars). For model selection, we trained and benchmarked 21 encoder architectures paired with a U-Net decoder. The selected model was integrated into a macOS application using CoreML inference on the Apple Neural Engine. Memory usage, thermal state and framerate were monitored on a continuous 81-minute console session. Photon-to-photon latency (from surgical field to the surgeon eye) was measured on the DaVinci® Xi using a microcontroller-phototransistor setup.

Results

FastViT-S12/Unet offered the highest accuracy among models meeting our inference-time budget (∼2 ms inference), achieving a Dice score of 95.1% (95% CI, 94.3-95.7) internally, 91.6% (95% CI, 90.2-92.9) and 90.2% (95% CI, 84.7-94.4) on externals tests, with a mean inference time of 1.89 ± 0.36 ms. The pipeline maintained 57.98 frames per second using 1.47 GB of unified memory with a nominal thermal state. On the surgical console, photon-to-photon median latency increased from 60 ms (Q1-Q3: 57.5-62.5) to 120.5 ms (Q1-Q3: 111-131) with the AR pipeline.

Conclusions

Real-time AR-RARP with DL-based desocclusion is achievable on low-cost hardware with an added latency remaining within ranges reported as technically acceptable for intraoperative use.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligence (AI) in surgeryimage guided surgeryrobotic surgeryurology

Autor:innen und Institutionen

  • Department of Urology, University Hospitals of Geneva, Geneva, Switzerland.
  • Division of Medical Information Sciences, University Hospitals of Geneva, Geneva, Switzerland.
  • Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland.
  • Departement of Surgery, University Hospitals of Geneva, Geneva, Switzerland.

ORCID-Profile: Vincent Benard, Olivier Windisch

Förderung

  • Foundation Nicod

Zitieren

APA

Benard, V., Windisch, O., Bjelogrlic, M., Turbé, H., Lovis, C., Diana, M., & Valerio, M. (2026). Deep-Learning Enhanced Augmented Reality for Robotic Radical Prostatectomy on Consumer Grade Hardware. Surgical Innovation. https://doi.org/10.1177/15533506261504154

Vancouver

Benard V, Windisch O, Bjelogrlic M, Turbé H, Lovis C, Diana M, et al. Deep-Learning Enhanced Augmented Reality for Robotic Radical Prostatectomy on Consumer Grade Hardware. Surgical Innovation. 2026. doi:10.1177/15533506261504154

BibTeX
@article{Benard2026,
  author = {Benard, Vincent and Windisch, Olivier and Bjelogrlic, Mina and Turbé, Hugues and Lovis, Christian and Diana, Michele and Valerio, Massimo},
  title = {Deep-Learning Enhanced Augmented Reality for Robotic Radical Prostatectomy on Consumer Grade Hardware},
  journal = {Surgical Innovation},
  year = {2026},
  doi = {10.1177/15533506261504154}
}
Kostenlos starten →