An augmented reality overlay for navigated prostatectomy using fiducial-free 2D–3D registration
International Journal of Computer Assisted Radiology and Surgery · 08.05.2025 · 20 (6) · 1265-1272
DOI: 10.1007/s11548-025-03374-5 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, online erschienen am 08.05.2025. Verfasst von Johannes Bender und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Translational Surgical Oncology, NCT/UCC Dresden, Dresden, Germany. johannes.bender@nct-dresden.de. Hauptthemen laut MEDLINE: Prostatectomy, Augmented Reality, Imaging, Three-Dimensional, Surgery, Computer-Assisted. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Bundesministerium für Bildung und Forschung, NCT Proof of Concept (PoC)-project NAIV.
Abstract (Originalsprache)
Purpose
Markerless navigation in minimally invasive surgery is still an unsolved challenge. Many proposed navigation systems for minimally invasive surgeries rely on stereoscopic images, while in clinical practice oftentimes monocular endoscopes are used. Combined with the lack of automatic video-based navigation systems for prostatectomies, this paper explores methods to tackle both research gaps at the same time for robot-assisted prostatectomies.
Methods
In order to realize a semi-automatic augmented reality overlay for navigated prostatectomy, the camera pose w.r.t. the prostate needs to be estimated. We developed a method where visual cues are drawn on top of the organ after an initial manual alignment, simultaneously creating matching landmarks on the 2D and 3D data. Starting from this key frame, the cues are then tracked in the endoscopic video. Both PnPRansac and differentiable rendering are then explored to perform 2D–3D registration for each frame.
Results
We performed experiments on synthetic and in vivo data. On synthetic data differentiable rendering can achieve a median target registration error of 6.11 mm. Both PnPRansac and differentiable rendering are feasible methods for 2D–3D registration.
Conclusion
We demonstrated a video-based markerless augmented reality overlay for navigated prostatectomy, using visual cues as an anchor.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Translational Surgical Oncology, NCT/UCC Dresden, Dresden, Germany. johannes.bender@nct-dresden.de.
- German Cancer Research Center (DKFZ), Heidelberg, Germany. johannes.bender@nct-dresden.de.
- Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany. johannes.bender@nct-dresden.de.
- Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Dresden, Germany. johannes.bender@nct-dresden.de.
- Department of Urology, University Hospital Carl Gustav Carus, TU Dresden, Dresden, Germany.
- Department of Radiology, University Hospital Carl Gustav Carus, TU Dresden, Dresden, Germany.
Förderung
- Bundesministerium für Bildung und Forschung
- NCT Proof of Concept (PoC)-project NAIV
Zitieren
APA
Bender, J., Kwe, J., Hoeh, B., Boehm, K., Platzek, I., Borkowetz, A., Speidel, S., & Pfeiffer, M. (2025). An augmented reality overlay for navigated prostatectomy using fiducial-free 2D–3D registration. International Journal of Computer Assisted Radiology and Surgery, 20(6), 1265-1272. https://doi.org/10.1007/s11548-025-03374-5
Vancouver
Bender J, Kwe J, Hoeh B, Boehm K, Platzek I, Borkowetz A, et al. An augmented reality overlay for navigated prostatectomy using fiducial-free 2D–3D registration. International Journal of Computer Assisted Radiology and Surgery. 2025;20(6):1265-1272. doi:10.1007/s11548-025-03374-5
BibTeX
@article{Bender2025,
author = {Bender, Johannes and Kwe, Jeremy and Hoeh, Benedikt and Boehm, Katharina and Platzek, Ivan and Borkowetz, Angelika and Speidel, Stefanie and Pfeiffer, Micha},
title = {An augmented reality overlay for navigated prostatectomy using fiducial-free 2D–3D registration},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2025},
volume = {20},
number = {6},
pages = {1265--1272},
doi = {10.1007/s11548-025-03374-5}
}
EUVIMED