Generalized Few‐Shot MM‐Former For Surgical Scene Panoptic Segmentation
Healthcare Technology Letters · 2025-01 · 12 (1)
DOI: 10.1049/htl2.70047 ↗Kurz gefasst
Fachartikel in der Zeitschrift Healthcare Technology Letters, online erschienen am 09.12.2025. Verfasst von Xiaoyan Zhang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Key Laboratory for Biomedical Engineering of Ministry of Education College of Biomedical Engineering and Instrument Science Zhejiang University Zheji…. Die Arbeit nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Key Research and Development Program of Zhejiang Province, National Key Research and Development Program of China.
Abstract (Originalsprache)
Panoptic segmentation is crucial for surgical scene understanding but remains a significant challenge. This is particularly due to the high cost of annotation, which often results in class imbalance in existing datasets, leading to poor performance on categories with limited samples. To address it, we proposed a generalized few‐shot MM‐former, which is a three‐stage framework: (1) We build surgical image‐text pairs from the CholecT50 dataset. Using these data, we fine‐tune the stable diffusion model to extract multi‐scale, image‐text fused representations. (2) We train an Mask2Former‐based panoptic segmentation decoder on the base classes with sufficient samples, and use it to transform the representations of each image to a set of mask proposals with category predictions. (3) We propose an N‐to‐M mask matching method. Given a small set of samples from N novel classes, we extract their features as guidance to match M mask proposals, enabling identification of all novel class objects in a single pass. Specifically, each matched proposal is updated with the most likely novel class, while the others keep original predictions. Finally, all proposals are merged to output the results. On CholecPanSeg, our newly built surgical panoptic dataset, the method achieves outstanding results under limited data, surpassing previous approaches.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Key Laboratory for Biomedical Engineering of Ministry of Education College of Biomedical Engineering and Instrument Science Zhejiang University Zhejiang China.
- The First Affiliated Hospital Zhejiang University School of Medicine Zhejiang China.
ORCID-Profile: Xiaoyan Zhang, Jiquan Liu
Förderung
- Key Research and Development Program of Zhejiang Province
- National Key Research and Development Program of China
Zitieren
APA
Zhang, X., Wu, L., Wang, Z., Feng, J., Zhu, Y., Zhou, Z., Tao, Y., Liu, J., & Duan, H. (2025). Generalized Few‐Shot MM‐Former For Surgical Scene Panoptic Segmentation. Healthcare Technology Letters, 12(1). https://doi.org/10.1049/htl2.70047
Vancouver
Zhang X, Wu L, Wang Z, Feng J, Zhu Y, Zhou Z, et al. Generalized Few‐Shot MM‐Former For Surgical Scene Panoptic Segmentation. Healthcare Technology Letters. 2025;12(1). doi:10.1049/htl2.70047
BibTeX
@article{Zhang2025,
author = {Zhang, Xiaoyan and Wu, Liming and Wang, Zhichen and Feng, Jingyi and Zhu, Yichen and Zhou, Ziyu and Tao, Ye and Liu, Jiquan and Duan, Huilong},
title = {Generalized Few‐Shot MM‐Former For Surgical Scene Panoptic Segmentation},
journal = {Healthcare Technology Letters},
year = {2025},
volume = {12},
number = {1},
doi = {10.1049/htl2.70047}
}
EUVIMED