RobustOVS: open-vocabulary segmentation with robustly semantic-assisted calibration
Multimedia Systems · 2026-06 · 32 (3)
DOI: 10.1007/s00530-026-02267-0 ↗Kurz gefasst
Fachartikel in der Zeitschrift Multimedia Systems, online erschienen am 10.03.2026. Verfasst von Ruihan Wang und 2 weiteren Autor:innen. Die Arbeit nennt 82 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Qingdao Natural Science Foundation, Textile Plus Joint Research Program of Qingdao University.
Abstract (Originalsprache)
Open-vocabulary semantic segmentation has emerged as a promising paradigm in image segmentation. Open-vocabulary segmentation models (OVS) typically leverage pre-trained vision–language models, such as CLIP, to perform semantic classification over masked regions. However, existing approaches often suffer from performance limitations when aligning visual representations with the unbounded semantic space of textual concepts. To address this challenge, we propose the Robust Open-Vocabulary Segmentation Model (RobustOVS), which preserves the generalization capability of CLIP while improving computational efficiency. Training open-vocabulary segmentation models usually requires substantial computational resources, which can be prohibitive for many research laboratories. RobustOVS alleviates this limitation through a streamlined and efficient network architecture that significantly reduces training overhead. The additional parameters introduced by RobustOVS can be trained and fine-tuned on a single GPU within 50 h, demonstrating its practicality for standard research environments. In RobustOVS, we incorporate a high-performance multi-scale feature pyramid network that extracts semantically rich representations by combining deformable convolutions with context-aware self-modulation. This design enables robust matching between masked image regions and noun-level concepts derived from image captions. Experimental results show that mask prompt fine-tuning consistently improves performance without modifying the original CLIP weights, while further enhancing models that are fully fine-tuned. We evaluate RobustOVS on several widely used open-vocabulary semantic segmentation benchmarks, where it achieves strong and stable performance across different tasks and datasets, while maintaining lower computational requirements compared with task-specific architectures.
Förderung
- National Natural Science Foundation of China
- Qingdao Natural Science Foundation
- Textile Plus Joint Research Program of Qingdao University
Zitieren
APA
Wang, R., Wang, G., & Liu, M. (2026). RobustOVS: open-vocabulary segmentation with robustly semantic-assisted calibration. Multimedia Systems, 32(3). https://doi.org/10.1007/s00530-026-02267-0
Vancouver
Wang R, Wang G, Liu M. RobustOVS: open-vocabulary segmentation with robustly semantic-assisted calibration. Multimedia Systems. 2026;32(3). doi:10.1007/s00530-026-02267-0
BibTeX
@article{Wang2026,
author = {Wang, Ruihan and Wang, Guodong and Liu, Mingtao},
title = {RobustOVS: open-vocabulary segmentation with robustly semantic-assisted calibration},
journal = {Multimedia Systems},
year = {2026},
volume = {32},
number = {3},
doi = {10.1007/s00530-026-02267-0}
}
EUVIMED