Vision and Language Reference for a Segment Anything Model for Few-Shot Segmentation
Journal of Imaging · 24.03.2026 · 12 (4) · 143
DOI: 10.3390/jimaging12040143 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Imaging, online erschienen am 24.03.2026. Verfasst von Kosuke Sakurai und 2 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Creative Science and Engineering, Waseda University, Tokyo 169-8555, Japan. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science, JSPS KAKENHI.
Abstract (Originalsprache)
Segment Anything Model (SAM)-based few-shot segmentation models traditionally rely solely on annotated reference images as prompts, which inherently limits their accuracy due to an over-reliance on visual cues and a lack of semantic context. This reliance leads to incorrect segmentation, where visually similar objects from different categories are incorrectly identified as the target object. We propose Vision and Language Reference Prompt into SAM (VLP-SAM), a novel few-shot segmentation model that integrates both visual information of reference images and semantic information of text labels into SAM. VLP-SAM introduces a vision-language model (VLM) with pixel–text matching into the prompt encoder for SAM, effectively leveraging textual semantic consistency while preserving SAM’s extensive segmentation knowledge. By incorporating task-specific structures such as an attention mask, our model achieves superior few-shot segmentation performance with only 1.4 M learnable parameters. Evaluations on PASCAL-5i and COCO-20i datasets demonstrate that VLP-SAM significantly outperforms previous methods by 6.8% and 9.3% in mIoU, respectively. Furthermore, VLP-SAM exhibits strong generalization across unseen objects and cross-domain scenarios, highlighting the robustness provided by textual semantic guidance. This study offers an effective and scalable framework for few-shot segmentation with multimodal prompts.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Graduate School of Creative Science and Engineering, Waseda University, Tokyo 169-8555, Japan.
- Institute of Data Science, Waseda University, Tokyo 169-8555, Japan.
- School of Creative Science and Engineering, Waseda University, Tokyo 169-8555, Japan.
ORCID-Profile: Kosuke Sakurai, Ryotaro Shimizu, Masayuki Goto
Förderung
- Japan Society for the Promotion of Science
- JSPS KAKENHI
Zitieren
APA
Sakurai, K., Shimizu, R., & Goto, M. (2026). Vision and Language Reference for a Segment Anything Model for Few-Shot Segmentation. Journal of Imaging, 12(4), 143. https://doi.org/10.3390/jimaging12040143
Vancouver
Sakurai K, Shimizu R, Goto M. Vision and Language Reference for a Segment Anything Model for Few-Shot Segmentation. Journal of Imaging. 2026;12(4):143. doi:10.3390/jimaging12040143
BibTeX
@article{Sakurai2026,
author = {Sakurai, Kosuke and Shimizu, Ryotaro and Goto, Masayuki},
title = {Vision and Language Reference for a Segment Anything Model for Few-Shot Segmentation},
journal = {Journal of Imaging},
year = {2026},
volume = {12},
number = {4},
pages = {143},
doi = {10.3390/jimaging12040143}
}
EUVIMED