Plug‐and‐play segment anything model improves nnUNet performance
Medical Physics · 2025-02 · 52 (2) · 899-912
DOI: 10.1002/mp.17481 ↗At a glance
Journal article in Medical Physics, published online on 2024-10-28. Written by Yunxiang Li and 4 co-authors. Institution: Department of Radiation Oncology, UT Southwestern Medical Center, Dallas, Texas, USA. Major MEDLINE topics: Image Processing, Computer-Assisted. Cited 48 times so far (Crossref); 30 references. The full text is freely available (open access, CC BY 4.0). Funded by: NIH HHS, NCI NIH HHS, NIBIB NIH HHS.
Abstract (original language)
Background
The automatic segmentation of medical images has widespread applications in modern clinical workflows. The Segment Anything Model (SAM), a recent development of foundational models in computer vision, has become a universal tool for image segmentation without the need for specific domain training. However, SAM's reliance on prompts necessitates human‐computer interaction during the inference process. Its performance on specific domains can also be limited without additional adaptation. In contrast, traditional models like nnUNet are designed to perform segmentation tasks automatically during inference and can work well for each specific domain, but they require extensive training on domain‐specific datasets.
Purpose
To leverage the advantages of both foundational and domain‐specific models and achieve fully automated segmentation with limited training samples, we propose nnSAM, which combines the robust feature extraction capabilities of SAM with the automatic configuration abilities of nnUNet to enhance the accuracy and robustness of medical image segmentation on small datasets.
Methods
We propose the nnSAM model for small sample medical image segmentation. We made optimizations for this goal via two main approaches: first, we integrated the feature extraction capabilities of SAM with the automatic configuration advantages of nnUNet, which enables robust feature extraction and domain‐specific adaptation on small datasets. Second, during the training process, we designed a boundary shape supervision loss based on level set functions and curvature calculations, enabling the model to learn anatomical shape priors from limited annotation data.
Results
We conducted quantitative and qualitative assessments on the performance of our proposed method on four segmentation tasks: brain white matter, liver, lung, and heart segmentation. Our method achieved the best performance across all tasks. Specifically, in brain white matter segmentation using 20 training samples, nnSAM achieved the highest DICE score of 82.77 ( 10.12) % and the lowest average surface distance (ASD) of 1.14 ( 1.03) mm, compared to nnUNet, which had a DICE score of 79.25 ( 17.24) % and an ASD of 1.36 ( 1.63) mm. A sample size study shows that the advantage of nnSAM becomes more prominent under fewer training samples.
Conclusions
A comprehensive evaluation of multiple small‐sample segmentation tasks demonstrates significant improvements in segmentation performance by nnSAM, highlighting the vast potential of small‐sample learning.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Radiation Oncology, UT Southwestern Medical Center, Dallas, Texas, USA.
- Department of Bioengineering, University of Washington, Seattle, Washington, USA.
Funding
- NIH HHS
- NCI NIH HHS
- NIBIB NIH HHS
- National Institutes of Health
Cite
APA
Li, Y., Jing, B., Li, Z., Wang, J., & Zhang, Y. (2025). Plug‐and‐play segment anything model improves nnUNet performance. Medical Physics, 52(2), 899-912. https://doi.org/10.1002/mp.17481
Vancouver
Li Y, Jing B, Li Z, Wang J, Zhang Y. Plug‐and‐play segment anything model improves nnUNet performance. Medical Physics. 2025;52(2):899-912. doi:10.1002/mp.17481
BibTeX
@article{Li2025,
author = {Li, Yunxiang and Jing, Bowen and Li, Zihan and Wang, Jing and Zhang, You},
title = {Plug‐and‐play segment anything model improves nnUNet performance},
journal = {Medical Physics},
year = {2025},
volume = {52},
number = {2},
pages = {899--912},
doi = {10.1002/mp.17481}
}
EUVIMED