EUVIMEDEUVIMED entdecken
Open Access

Laparoscopic Projection Mapping of the Liver Portal Segment, Based on Augmented Reality Combined With Artificial Intelligence, for Laparoscopic Anatomical Liver Resection

Meidai Kasai, Hideaki Uchiyama, Tsukasa Aihara, Shinichi Ikuta, Naoki Yamanaka

Cureus · 07.11.2023

DOI: 10.7759/cureus.48450 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cureus, erschienen am 07.11.2023. Verfasst von Meidai Kasai und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Surgery, Meiwa Hospital, Nishinomiya, JPN. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Hepatocellular carcinoma causes intrahepatic metastasis via the trans-portal vein. Thus, appropriate mapping of portal segments is necessary for laparoscopic anatomical liver resection. However, because of the difficulty in identifying tactile sensations and the limited surgical view of laparoscopy, augmented reality (AR) has recently been utilized in laparoscopic liver surgery to identify the tumor, vessels, and portal segments. Moreover, artificial intelligence (AI) has been employed to identify landmarks in two-dimensional (2D) images because of concerns regarding the accuracy of superimposing a three-dimensional (3D) model onto a 2D laparoscopic image. In this study, we report an AR-based projection mapping method of portal segments superimposing preoperative 3D models assisted by AI in laparoscopic surgery. The liver silhouette in laparoscopic images should be detected to superimpose 3D models. Labeled liver silhouettes were obtained from 380 images in surgical videos as learning images to implement AI-based silhouette detection. To implement this technique, we used Detectron2, a PyTorch-based object detection library by Facebook AI Research (Now, Meta AI, Menlo Park, California, United States). In the videos, the liver edges were displayed as green outlines according to AI. Additionally, 3D liver models with segmental mapping were generated using the open-source software 3D Slicer from computed tomography images. For AR display, we utilized the model target function of Vuforia SDK (PTC, Inc., Boston, Massachusetts, United States), an industrial AR library with silhouette-based AR display. Lastly, we merged the AI output video with a 3D model in Unity (Unity Software Inc., San Francisco, California, United States) to establish the projection mapping of the portal segment on 2D surgical images. The accuracy was assessed by measuring the maximum error between the liver edges of laparoscopic images and 3D liver silhouettes in five surgical videos. The maximum error between liver edges and 3D model silhouettes ranged from 4 mm to 22 mm in the AI-based approach and 12 mm to 55 mm in the non-AI-based approach. Meanwhile, the mean error was 14.5 and 31.2 mm in the AI-based and non-AI-based approaches, respectively. Despite camera movement, 3D AR displays were maintained. Thus, our AI-assisted projection mapping of the portal segment could offer a new approach for laparoscopic anatomical liver resection.

Schlagwörter

Stichwörter der Autor:innen

3d surgical navigationai & robotics in healthcareanatomical liver resectionartificial intelligence in surgeryaugmented realityaugmented reality surgical navigationfuture technologies in minimally invasive surgery and artificial intelligencelaparoscopic liver surgerysurgeryvirtual augmented reality

Autor:innen und Institutionen

  • Department of Surgery, Meiwa Hospital, Nishinomiya, JPN.
  • Graduate School of Science and Technology, Nara Institute of Science and Technology, Ikoma, JPN.

Zitieren

APA

Kasai, M., Uchiyama, H., Aihara, T., Ikuta, S., & Yamanaka, N. (2023). Laparoscopic Projection Mapping of the Liver Portal Segment, Based on Augmented Reality Combined With Artificial Intelligence, for Laparoscopic Anatomical Liver Resection. Cureus. https://doi.org/10.7759/cureus.48450

Vancouver

Kasai M, Uchiyama H, Aihara T, Ikuta S, Yamanaka N. Laparoscopic Projection Mapping of the Liver Portal Segment, Based on Augmented Reality Combined With Artificial Intelligence, for Laparoscopic Anatomical Liver Resection. Cureus. 2023. doi:10.7759/cureus.48450

BibTeX
@article{Kasai2023,
  author = {Kasai, Meidai and Uchiyama, Hideaki and Aihara, Tsukasa and Ikuta, Shinichi and Yamanaka, Naoki},
  title = {Laparoscopic Projection Mapping of the Liver Portal Segment, Based on Augmented Reality Combined With Artificial Intelligence, for Laparoscopic Anatomical Liver Resection},
  journal = {Cureus},
  year = {2023},
  doi = {10.7759/cureus.48450}
}
Kostenlos starten →