Unsupervised Pre-Training With Language-Vision Prompts for Low-Data Instance Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-10 · 47 (10) · 8642-8657
DOI: 10.1109/tpami.2025.3579469 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-10. Verfasst von Dingwen Zhang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 62 Literaturangaben. Gefördert von: National Science and Technology Major Project, National Natural Science Foundation of China.
Abstract (Originalsprache)
In recent times, following the paradigm of DETR (DEtection TRansformer), query-based end-to-end instance segmentation (QEIS) methods have exhibited superior performance compared to CNN-based models, particularly when trained on large-scale datasets. Nevertheless, the effectiveness of these QEIS methods diminishes significantly when confronted with limited training data. This limitation arises from their reliance on substantial data volumes to effectively train the pivotal queries/kernels that are essential for acquiring localization and shape priors. To address this problem, we propose a novel method for unsupervised pre-training in low-data regimes. Inspired by the recently successful prompting technique, we introduce a new method, Unsupervised Pre-training with Language-Vision Prompts (UPLVP), which improves QEIS models' instance segmentation by bringing language-vision prompts to queries/kernels. Our method consists of three parts: (1) Masks Proposal: Utilizes language-vision models to generate pseudo masks based on unlabeled images. (2) Prompt-Kernel Matching: Converts pseudo masks into prompts and injects the best-matched localization and shape features to their corresponding kernels. (3) Kernel Supervision: Formulates supervision for pre-training at the kernel level to ensure robust learning. With the help of our pre-training method, QEIS models can converge faster and perform better than CNN-based models in low-data regimes. Experimental evaluations conducted on MS COCO, Cityscapes, and CTW1500 datasets indicate that the QEIS models' performance can be significantly improved when pre-trained with our method.
Förderung
- National Science and Technology Major Project
- National Natural Science Foundation of China
Zitieren
APA
Zhang, D., Li, H., He, D., Liu, N., Cheng, L., Wang, J., & Han, J. (2025). Unsupervised Pre-Training With Language-Vision Prompts for Low-Data Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(10), 8642-8657. https://doi.org/10.1109/tpami.2025.3579469
Vancouver
Zhang D, Li H, He D, Liu N, Cheng L, Wang J, et al. Unsupervised Pre-Training With Language-Vision Prompts for Low-Data Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(10):8642-8657. doi:10.1109/tpami.2025.3579469
BibTeX
@article{Zhang2025,
author = {Zhang, Dingwen and Li, Hao and He, Diqi and Liu, Nian and Cheng, Lechao and Wang, Jingdong and Han, Junwei},
title = {Unsupervised Pre-Training With Language-Vision Prompts for Low-Data Instance Segmentation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {10},
pages = {8642--8657},
doi = {10.1109/tpami.2025.3579469}
}
EUVIMED