SeeSaw: Learning Soft Tissue Deformation From Laparoscopy Videos With GNNs
IEEE Transactions on Biomedical Engineering · 2024
DOI: 10.1109/tbme.2024.3424771 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Biomedical Engineering, 2024. Verfasst von Reuben Docea und 9 weiteren Autor:innen. Hauptthemen laut MEDLINE: Laparoscopy, Neural Networks, Computer, Surgery, Computer-Assisted, Video Recording. Die Arbeit wurde bisher 5-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
A major challenge in image-guided laparoscopic surgery is that structures of interest often deform and go, even if only momentarily, out of view. Methods which rely on having an up-to-date impression of those structures, such as registration or localisation, are undermined in these circumstances. This is particularly true for soft-tissue structures that continually change shape - in registration, they must often be re-mapped. Furthermore, methods which require 'revisiting' of previously seen areas cannot in principle function reliably in dynamic contexts, drastically weakening their uptake in the operating room. We present a novel approach for learning to estimate the deformed states of previously seen soft tissue surfaces from currently observable regions, using a combined approach that includes a Graph Neural Network (GNN). The training data is based on stereo laparoscopic surgery videos, generated semi-automatically with minimal labelling effort. Trackable segments are first identified using a feature detection algorithm, from which surface meshes are produced using depth estimation and delaunay triangulation. We show the method can predict the displacements of previously visible soft tissue structures connected to currently visible regions with observed displacements, both on patient data and porcine data. Our innovative approach learns to compensate non-rigidity in abdominal endoscopic scenes directly from stereo laparoscopic videos through targeting a new problem formulation, and stands to benefit a variety of target applications in dynamic environments.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Zitieren
APA
Docea, R., Xu, J., Ling, W., Jenke, A. C., Kolbinger, F. R., Distler, M., Riediger, C., Weitz, J., Speidel, S., & Pfeiffer, M. (2024). SeeSaw: Learning Soft Tissue Deformation From Laparoscopy Videos With GNNs. IEEE Transactions on Biomedical Engineering. https://doi.org/10.1109/tbme.2024.3424771
Vancouver
Docea R, Xu J, Ling W, Jenke AC, Kolbinger FR, Distler M, et al. SeeSaw: Learning Soft Tissue Deformation From Laparoscopy Videos With GNNs. IEEE Transactions on Biomedical Engineering. 2024. doi:10.1109/tbme.2024.3424771
BibTeX
@article{Docea2024,
author = {Docea, Reuben and Xu, Jinjing and Ling, Wei and Jenke, Alexander C and Kolbinger, Fiona R and Distler, Marius and Riediger, Carina and Weitz, Jurgen and Speidel, Stefanie and Pfeiffer, Micha},
title = {SeeSaw: Learning Soft Tissue Deformation From Laparoscopy Videos With GNNs},
journal = {IEEE Transactions on Biomedical Engineering},
year = {2024},
doi = {10.1109/tbme.2024.3424771}
}
EUVIMED