Foundation Model-Guided Gaussian Splatting for 4D Reconstruction of Deformable Tissues
IEEE Transactions on Medical Imaging · 2025-06 · 44 (6) · 2672-2682
DOI: 10.1109/tmi.2025.3545183 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025-06. Verfasst von Yifan Liu und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Imaging, Three-Dimensional. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Hong Kong Research Grants Council (RGC)/General Research Fund.
Abstract (Originalsprache)
Reconstructing deformable anatomical structures from endoscopic videos is a pivotal and promising research topic that can enable advanced surgical applications and improve patient outcomes. While existing surgical scene reconstruction methods have made notable progress, they often suffer from slow rendering speeds due to using neural radiance fields, limiting their practical viability in real-world applications. To overcome this bottleneck, we propose EndoGaussian, a framework that integrates the strengths of 3D Gaussian Splatting representations, allowing for high-fidelity tissue reconstruction, efficient training, and real-time rendering. Specifically, we dedicate a Foundation Model-driven Initialization (FMI) module, which distills 3D cues from multiple vision foundation models (VFMs) to swiftly construct the preliminary scene structure for Gaussian initialization. Then, a Spatio-temporal Gaussian Tracking (SGT) is designed, efficiently modeling scene dynamics using the multi-scale HexPlane with spatio-temporal priors. Furthermore, to improve the dynamics modeling ability for scenes with large deformation, EndoGaussian integrates Motion-aware Frame Synthesis (MFS) to adaptively synthesize new frames as extra training constraints. Experimental results on public datasets demonstrate EndoGaussian's efficacy against prior state-of-the-art methods, including superior rendering speed (168 FPS, real-time), enhanced rendering quality (38.555 PSNR), and reduced training overhead (within 2 min/scene). These results underscore EndoGaussian's potential to significantly advance intraoperative surgery applications, paving the way for more accurate and efficient real-time surgical guidance and decision-making in clinical scenarios. Code is available at: https://github.com/CUHK-AIM-Group/EndoGaussian.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Natural Science Foundation of China
- Hong Kong Research Grants Council (RGC)/General Research Fund
Zitieren
APA
Liu, Y., Li, C., Liu, H., Yang, C., & Yuan, Y. (2025). Foundation Model-Guided Gaussian Splatting for 4D Reconstruction of Deformable Tissues. IEEE Transactions on Medical Imaging, 44(6), 2672-2682. https://doi.org/10.1109/tmi.2025.3545183
Vancouver
Liu Y, Li C, Liu H, Yang C, Yuan Y. Foundation Model-Guided Gaussian Splatting for 4D Reconstruction of Deformable Tissues. IEEE Transactions on Medical Imaging. 2025;44(6):2672-2682. doi:10.1109/tmi.2025.3545183
BibTeX
@article{Liu2025,
author = {Liu, Yifan and Li, Chenxin and Liu, Hengyu and Yang, Chen and Yuan, Yixuan},
title = {Foundation Model-Guided Gaussian Splatting for 4D Reconstruction of Deformable Tissues},
journal = {IEEE Transactions on Medical Imaging},
year = {2025},
volume = {44},
number = {6},
pages = {2672--2682},
doi = {10.1109/tmi.2025.3545183}
}
EUVIMED