Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields
Biomedical Optics Express · 01.06.2024 · 15 (6) · 3914
DOI: 10.1364/boe.521612 ↗Kurz gefasst
Fachartikel in der Zeitschrift Biomedical Optics Express, online erschienen am 23.05.2024. Verfasst von Zhibao Qin und 5 weiteren Autor:innen. Beteiligte Einrichtung: Yunnan Key Laboratory of Opto-electronic Information Technology, Yunnan Normal University, Kunming 650500, China. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: National Natural Science Foundation of China, Yunnan Outstanding Youth Fund.
Abstract (Originalsprache)
Virtual surgical training is crucial for enhancing minimally invasive surgical skills. Traditional geometric reconstruction methods based on medical CT/MRI images often fall short in providing color information, which is typically generated through pseudo-coloring or artistic rendering. To simultaneously reconstruct both the geometric shape and appearance information of organs, we propose a novel organ model reconstruction network called Endoscope-NeSRF. This network jointly leverages neural radiance fields and Signed Distance Function (SDF) to reconstruct a textured geometric model of the organ of interest from multi-view photometric images acquired by an endoscope. The prior knowledge of the inverse correlation between the distance from the light source to the object and the radiance improves the real physical properties of the organ. The dilated mask further refines the appearance and geometry at the organ's edges. We also proposed a highlight adaptive optimization strategy to remove highlights caused by the light source during the acquisition process, thereby preventing the reconstruction results in areas previously affected by highlights from turning white. Finally, the real-time realistic rendering of the organ model is achieved by combining the inverse rendering and Bidirectional Reflectance Distribution Function (BRDF) rendering methods. Experimental results show that our method closely matches the Instant-NGP method in appearance reconstruction, outperforming other state-of-the-art methods, and stands as the superior method in terms of geometric reconstruction. Our method obtained a detailed geometric model and realistic appearance, providing a realistic visual sense for virtual surgical simulation, which is important for medical training.
Autor:innen und Institutionen
- Yunnan Key Laboratory of Opto-electronic Information Technology, Yunnan Normal University, Kunming 650500, China.
- Department of Thoracic Surgery, Institute of The First People's Hospital of Yunnan Province, Kunming 650500, China.
ORCID-Profile: Zhibao Qin, Yonghang Tai
Förderung
- National Natural Science Foundation of China
- Yunnan Outstanding Youth Fund
Zitieren
APA
Qin, Z., Chen, Q., Qian, K., Zheng, Q., Shi, J., & Tai, Y. (2024). Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields. Biomedical Optics Express, 15(6), 3914. https://doi.org/10.1364/boe.521612
Vancouver
Qin Z, Chen Q, Qian K, Zheng Q, Shi J, Tai Y. Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields. Biomedical Optics Express. 2024;15(6):3914. doi:10.1364/boe.521612
BibTeX
@article{Qin2024,
author = {Qin, Zhibao and Chen, Qi and Qian, Kai and Zheng, Qinhong and Shi, Junsheng and Tai, Yonghang},
title = {Enhancing endoscopic scene reconstruction with color-aware inverse rendering through neural SDF and radiance fields},
journal = {Biomedical Optics Express},
year = {2024},
volume = {15},
number = {6},
pages = {3914},
doi = {10.1364/boe.521612}
}
EUVIMED