EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery

Alexander Saikia, Chiara Di Vece, Zhehua Mao, Sierra Bonilla, Chloe He, Joao Ramalhinho, Tobias Czempiel, Sophia Bano, Danail Stoyanov

International Journal of Computer Assisted Radiology and Surgery · 27.04.2026 · 21 (5) · 1061-1069

DOI: 10.1007/s11548-026-03633-z ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Assisted Radiology and Surgery, online erschienen am 27.04.2026. Verfasst von Alexander Saikia und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Physics and Biomedical Engineering, UCL Hawkes Institute, University College London, London, WC1E 6BT, UK. alexander.saikia.21@…. Hauptthemen laut MEDLINE: Imaging, Three-Dimensional, Hyperspectral Imaging, Minimally Invasive Surgical Procedures, Laparoscopy. Die Arbeit nennt 20 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wellcome, EPSRC Centre for Doctoral Training in Medical Imaging, UK Research and Innovation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

3D reconstruction in minimally invasive surgery (MIS) enables enhanced surgical guidance through improved visualisation, tool tracking, and augmented reality. However, traditional RGB-based keypoint detection and matching pipelines struggle with surgical challenges, such as poor texture and complex illumination. We investigate whether using snapshot hyperspectral imaging (HSI) can provide improved results on keypoint detection and matching surgical scenes.

Methods

We developed HyKey, a hyperspectral keypoint detection and description model made up of a hybrid 3D-2D convolutional neural network that jointly extracts spatial-spectral features from HSI. The model was trained using synthetic homographic augmentation and epipolar geometry constraints on a robotically acquired dual-camera RGB-HSI laparoscopic dataset of ex vivo organs with calibrated camera poses. We benchmarked performance against established RGB-based methods, including SuperPoint and ALIKE.

Results

Our HSI-based model outperformed RGB baselines on registered RGB frames, achieving 96.62% mean matching accuracy and 67.18% mean average accuracy at 10 $$^\circ $$ ∘ on pose estimation, demonstrating consistent improvements across multiple evaluation metrics.

Conclusion

Integrating spectral information from an HSI cube offers a promising approach for robust monocular 3D reconstruction in MIS, addressing limitations of texture-poor surgical environments through enhanced spectral-spatial feature discrimination. Our model and dataset are available at https://github.com/alexsaikia/HyKey-Hyperspectral-Keypoint-Detection.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Imaging, Three-DimensionalHyperspectral ImagingMinimally Invasive Surgical ProceduresLaparoscopySurgery, Computer-AssistedHumansConvolutional Neural Networks

Stichwörter der Autor:innen

3D reconstructionHyperspectral imagingKeypoint detection and descriptionMinimally invasive surgery

Autor:innen und Institutionen

  • Department of Medical Physics and Biomedical Engineering, UCL Hawkes Institute, University College London, London, WC1E 6BT, UK. alexander.saikia.21@ucl.ac.uk.
  • Department of Computer Science, University College London, London, WC1E 6BT, UK. alexander.saikia.21@ucl.ac.uk.
  • EnAcuity Ltd., London, EC2A 4NE, UK. alexander.saikia.21@ucl.ac.uk.
  • Department of Medical Physics and Biomedical Engineering, UCL Hawkes Institute, University College London, London, WC1E 6BT, UK.
  • Department of Computer Science, University College London, London, WC1E 6BT, UK.
  • EnAcuity Ltd., London, EC2A 4NE, UK.

Förderung

  • Wellcome
  • EPSRC Centre for Doctoral Training in Medical Imaging
  • UK Research and Innovation

Zitieren

APA

Saikia, A., Di Vece, C., Mao, Z., Bonilla, S., He, C., Ramalhinho, J., Czempiel, T., Bano, S., & Stoyanov, D. (2026). HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery. International Journal of Computer Assisted Radiology and Surgery, 21(5), 1061-1069. https://doi.org/10.1007/s11548-026-03633-z

Vancouver

Saikia A, Di Vece C, Mao Z, Bonilla S, He C, Ramalhinho J, et al. HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery. International Journal of Computer Assisted Radiology and Surgery. 2026;21(5):1061-1069. doi:10.1007/s11548-026-03633-z

BibTeX
@article{Saikia2026,
  author = {Saikia, Alexander and Di Vece, Chiara and Mao, Zhehua and Bonilla, Sierra and He, Chloe and Ramalhinho, Joao and Czempiel, Tobias and Bano, Sophia and Stoyanov, Danail},
  title = {HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery},
  journal = {International Journal of Computer Assisted Radiology and Surgery},
  year = {2026},
  volume = {21},
  number = {5},
  pages = {1061--1069},
  doi = {10.1007/s11548-026-03633-z}
}
Kostenlos starten →