HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery
International Journal of Computer Assisted Radiology and Surgery · 27.04.2026 · 21 (5) · 1061-1069
DOI: 10.1007/s11548-026-03633-z ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, online erschienen am 27.04.2026. Verfasst von Alexander Saikia und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Physics and Biomedical Engineering, UCL Hawkes Institute, University College London, London, WC1E 6BT, UK. alexander.saikia.21@…. Hauptthemen laut MEDLINE: Imaging, Three-Dimensional, Hyperspectral Imaging, Minimally Invasive Surgical Procedures, Laparoscopy. Die Arbeit nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wellcome, EPSRC Centre for Doctoral Training in Medical Imaging, UK Research and Innovation.
Abstract (Originalsprache)
Purpose
3D reconstruction in minimally invasive surgery (MIS) enables enhanced surgical guidance through improved visualisation, tool tracking, and augmented reality. However, traditional RGB-based keypoint detection and matching pipelines struggle with surgical challenges, such as poor texture and complex illumination. We investigate whether using snapshot hyperspectral imaging (HSI) can provide improved results on keypoint detection and matching surgical scenes.
Methods
We developed HyKey, a hyperspectral keypoint detection and description model made up of a hybrid 3D-2D convolutional neural network that jointly extracts spatial-spectral features from HSI. The model was trained using synthetic homographic augmentation and epipolar geometry constraints on a robotically acquired dual-camera RGB-HSI laparoscopic dataset of ex vivo organs with calibrated camera poses. We benchmarked performance against established RGB-based methods, including SuperPoint and ALIKE.
Results
Our HSI-based model outperformed RGB baselines on registered RGB frames, achieving 96.62% mean matching accuracy and 67.18% mean average accuracy at 10 $$^\circ $$ ∘ on pose estimation, demonstrating consistent improvements across multiple evaluation metrics.
Conclusion
Integrating spectral information from an HSI cube offers a promising approach for robust monocular 3D reconstruction in MIS, addressing limitations of texture-poor surgical environments through enhanced spectral-spatial feature discrimination. Our model and dataset are available at https://github.com/alexsaikia/HyKey-Hyperspectral-Keypoint-Detection.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medical Physics and Biomedical Engineering, UCL Hawkes Institute, University College London, London, WC1E 6BT, UK. alexander.saikia.21@ucl.ac.uk.
- Department of Computer Science, University College London, London, WC1E 6BT, UK. alexander.saikia.21@ucl.ac.uk.
- EnAcuity Ltd., London, EC2A 4NE, UK. alexander.saikia.21@ucl.ac.uk.
- Department of Medical Physics and Biomedical Engineering, UCL Hawkes Institute, University College London, London, WC1E 6BT, UK.
- Department of Computer Science, University College London, London, WC1E 6BT, UK.
- EnAcuity Ltd., London, EC2A 4NE, UK.
Förderung
- Wellcome
- EPSRC Centre for Doctoral Training in Medical Imaging
- UK Research and Innovation
Zitieren
APA
Saikia, A., Di Vece, C., Mao, Z., Bonilla, S., He, C., Ramalhinho, J., Czempiel, T., Bano, S., & Stoyanov, D. (2026). HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery. International Journal of Computer Assisted Radiology and Surgery, 21(5), 1061-1069. https://doi.org/10.1007/s11548-026-03633-z
Vancouver
Saikia A, Di Vece C, Mao Z, Bonilla S, He C, Ramalhinho J, et al. HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery. International Journal of Computer Assisted Radiology and Surgery. 2026;21(5):1061-1069. doi:10.1007/s11548-026-03633-z
BibTeX
@article{Saikia2026,
author = {Saikia, Alexander and Di Vece, Chiara and Mao, Zhehua and Bonilla, Sierra and He, Chloe and Ramalhinho, Joao and Czempiel, Tobias and Bano, Sophia and Stoyanov, Danail},
title = {HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery},
journal = {International Journal of Computer Assisted Radiology and Surgery},
year = {2026},
volume = {21},
number = {5},
pages = {1061--1069},
doi = {10.1007/s11548-026-03633-z}
}
EUVIMED