EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality

Constantin Kleinbeck, Han Zhang, Benjamin D Killeen, Daniel Roth, Mathias Unberath

International Journal of Computer Assisted Radiology and Surgery · 2024

DOI: 10.1007/s11548-024-03143-w ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, 2024. Verfasst von Constantin Kleinbeck und 4 weiteren Autor:innen. Beteiligte Einrichtung: TUM School of Medicine and Health, Department Clinical Medicine, Orthopedics, Technical University of Munich, Munich, Germany. constantin.kleinbeck@t…. Hauptthemen laut MEDLINE: Virtual Reality, Operating Rooms. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, Digital Health and Innovation Platform, Link Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

Specialized robotic and surgical tools are increasing the complexity of operating rooms (ORs), requiring elaborate preparation especially when techniques or devices are to be used for the first time. Spatial planning can improve efficiency and identify procedural obstacles ahead of time, but real ORs offer little availability to optimize space utilization. Methods for creating reconstructions of physical setups, i.e., digital twins, are needed to enable immersive spatial planning of such complex environments in virtual reality.

Methods

We present a neural rendering-based method to create immersive digital twins of complex medical environments and devices from casual video capture that enables spatial planning of surgical scenarios. To evaluate our approach we recreate two operating rooms and ten objects through neural reconstruction, then conduct a user study with 21 graduate students carrying out planning tasks in the resulting virtual environment. We analyze task load, presence, perceived utility, plus exploration and interaction behavior compared to low visual complexity versions of the same environments.

Results

Results show significantly increased perceived utility and presence using the neural reconstruction-based environments, combined with higher perceived workload and exploratory behavior. There’s no significant difference in interactivity.

Conclusion

We explore the feasibility of using modern reconstruction techniques to create digital twins of complex medical environments and objects. Without requiring expert knowledge or specialized hardware, users can create, explore and interact with objects in virtual environments. Results indicate benefits like high perceived utility while being technically approachable, which may indicate promise of this approach for spatial planning and beyond.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Virtual RealityOperating RoomsHumansUser-Computer InterfaceFemaleMaleAdultFeasibility StudiesRobotic Surgical Procedures

Stichwörter der Autor:innen

Human–computer interactionMachine learningMixed realityNeRFRobotic surgery

Autor:innen und Institutionen

  • TUM School of Medicine and Health, Department Clinical Medicine, Orthopedics, Technical University of Munich, Munich, Germany. constantin.kleinbeck@tum.de.
  • Johns Hopkins University, Baltimore, MD, USA. constantin.kleinbeck@tum.de.
  • Friedrich-Alexander-Universität, Erlangen, Germany. constantin.kleinbeck@tum.de.
  • Johns Hopkins University, Baltimore, MD, USA.
  • TUM School of Medicine and Health, Department Clinical Medicine, Orthopedics, Technical University of Munich, Munich, Germany.

Förderung

  • National Science Foundation
  • Digital Health and Innovation Platform
  • Link Foundation

Zitieren

APA

Kleinbeck, C., Zhang, H., Killeen, B. D., Roth, D., & Unberath, M. (2024). Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-024-03143-w

Vancouver

Kleinbeck C, Zhang H, Killeen BD, Roth D, Unberath M. Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality. International Journal of Computer Assisted Radiology and Surgery. 2024. doi:10.1007/s11548-024-03143-w

BibTeX
@article{Kleinbeck2024,
  author = {Kleinbeck, Constantin and Zhang, Han and Killeen, Benjamin D and Roth, Daniel and Unberath, Mathias},
  title = {Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality},
  journal = {International Journal of Computer Assisted Radiology and Surgery},
  year = {2024},
  doi = {10.1007/s11548-024-03143-w}
}
Kostenlos starten →