UpGen: Unleashing Potential of Foundation Models for Training-Free Camouflage Detection via Generative Models
IEEE Transactions on Image Processing · 2025 · 34 · 5400-5413
DOI: 10.1109/tip.2025.3599101 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Ji Du und 8 weiteren Autor:innen. Hauptthemen laut MEDLINE: Image Processing, Computer-Assisted. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 110 Literaturangaben. Gefördert von: Hong Kong Polytechnic University, PolyU Research Institute for Sports Science and Technology, Tianjin Science and Technology Major Project.
Abstract (Originalsprache)
Camouflaged Object Detection (COD) aims to segment objects resembling their environment. To address the challenges of extensive annotations and complex optimizations in supervised learning, recent prompt-based segmentation methods excavate insightful prompts from Large Vision-Language Models (LVLMs) and refine them using various foundation models. These are subsequently fed into the Segment Anything Model (SAM) for segmentation. However, due to the hallucinations of LVLMs and insufficient image-prompt interactions during the refinement stage, these prompts often struggle to capture well-established class differentiation and localization of camouflaged objects, resulting in performance degradation. To provide SAM with more informative prompts, we present UpGen, a pipeline that prompts SAM with generative prompts without requiring training, marking a novel integration of generative models with LVLMs. Specifically, we propose the Multi-Student-Single-Teacher (MSST) knowledge integration framework to alleviate hallucinations of LVLMs. This framework integrates insights from multiple sources to enhance the classification of camouflaged objects. To enhance interactions during the prompt refinement stage, we are the first to leverage generative models on real camouflage images to produce SAM-style prompts without fine-tuning. By capitalizing on the unique learning mechanism and structure of generative models, we effectively enable image-prompt interactions and generate highly informative prompts for SAM. Our extensive experiments demonstrate that UpGen outperforms weakly-supervised models and its SAM-based counterparts. We also integrate our framework into existing weakly-supervised methods to generate pseudo-labels, resulting in consistent performance gains. Moreover, with minor adjustments, UpGen shows promising results in open-vocabulary COD, referring COD, salient object detection, marine animal segmentation, and transparent object segmentation.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- Hong Kong Polytechnic University
- PolyU Research Institute for Sports Science and Technology
- Tianjin Science and Technology Major Project
Zitieren
APA
Du, J., Wu, J., Kong, D., Liang, W., Hao, F., Xu, J., Wang, B., Wang, G., & Li, P. (2025). UpGen: Unleashing Potential of Foundation Models for Training-Free Camouflage Detection via Generative Models. IEEE Transactions on Image Processing, 34, 5400-5413. https://doi.org/10.1109/tip.2025.3599101
Vancouver
Du J, Wu J, Kong D, Liang W, Hao F, Xu J, et al. UpGen: Unleashing Potential of Foundation Models for Training-Free Camouflage Detection via Generative Models. IEEE Transactions on Image Processing. 2025;34:5400-5413. doi:10.1109/tip.2025.3599101
BibTeX
@article{Du2025,
author = {Du, Ji and Wu, Jiesheng and Kong, Desheng and Liang, Weiyun and Hao, Fangwei and Xu, Jing and Wang, Bin and Wang, Guiling and Li, Ping},
title = {UpGen: Unleashing Potential of Foundation Models for Training-Free Camouflage Detection via Generative Models},
journal = {IEEE Transactions on Image Processing},
year = {2025},
volume = {34},
pages = {5400--5413},
doi = {10.1109/tip.2025.3599101}
}
EUVIMED