Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality
International Journal of Computer Assisted Radiology and Surgery · 2024
DOI: 10.1007/s11548-024-03143-w ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, 2024. Verfasst von Constantin Kleinbeck und 4 weiteren Autor:innen. Beteiligte Einrichtung: TUM School of Medicine and Health, Department Clinical Medicine, Orthopedics, Technical University of Munich, Munich, Germany. constantin.kleinbeck@t…. Hauptthemen laut MEDLINE: Virtual Reality, Operating Rooms. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, Digital Health and Innovation Platform, Link Foundation.
Abstract (Originalsprache)
Purpose
Specialized robotic and surgical tools are increasing the complexity of operating rooms (ORs), requiring elaborate preparation especially when techniques or devices are to be used for the first time. Spatial planning can improve efficiency and identify procedural obstacles ahead of time, but real ORs offer little availability to optimize space utilization. Methods for creating reconstructions of physical setups, i.e., digital twins, are needed to enable immersive spatial planning of such complex environments in virtual reality.
Methods
We present a neural rendering-based method to create immersive digital twins of complex medical environments and devices from casual video capture that enables spatial planning of surgical scenarios. To evaluate our approach we recreate two operating rooms and ten objects through neural reconstruction, then conduct a user study with 21 graduate students carrying out planning tasks in the resulting virtual environment. We analyze task load, presence, perceived utility, plus exploration and interaction behavior compared to low visual complexity versions of the same environments.
Results
Results show significantly increased perceived utility and presence using the neural reconstruction-based environments, combined with higher perceived workload and exploratory behavior. There’s no significant difference in interactivity.
Conclusion
We explore the feasibility of using modern reconstruction techniques to create digital twins of complex medical environments and objects. Without requiring expert knowledge or specialized hardware, users can create, explore and interact with objects in virtual environments. Results indicate benefits like high perceived utility while being technically approachable, which may indicate promise of this approach for spatial planning and beyond.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- TUM School of Medicine and Health, Department Clinical Medicine, Orthopedics, Technical University of Munich, Munich, Germany. constantin.kleinbeck@tum.de.
- Johns Hopkins University, Baltimore, MD, USA. constantin.kleinbeck@tum.de.
- Friedrich-Alexander-Universität, Erlangen, Germany. constantin.kleinbeck@tum.de.
- Johns Hopkins University, Baltimore, MD, USA.
- TUM School of Medicine and Health, Department Clinical Medicine, Orthopedics, Technical University of Munich, Munich, Germany.
Förderung
- National Science Foundation
- Digital Health and Innovation Platform
- Link Foundation
Zitieren
APA
Kleinbeck, C., Zhang, H., Killeen, B. D., Roth, D., & Unberath, M. (2024). Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-024-03143-w
Vancouver
Kleinbeck C, Zhang H, Killeen BD, Roth D, Unberath M. Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality. International Journal of Computer Assisted Radiology and Surgery. 2024. doi:10.1007/s11548-024-03143-w
BibTeX
@article{Kleinbeck2024,
author = {Kleinbeck, Constantin and Zhang, Han and Killeen, Benjamin D and Roth, Daniel and Unberath, Mathias},
title = {Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2024},
doi = {10.1007/s11548-024-03143-w}
}
EUVIMED