EUVIMEDEUVIMED entdecken
Crossref

Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception

Junjie Wang, Keyu Chen, Yulin Li, Bin Chen, Hengshuang Zhao, Xiaojuan Qi, Zhuotao Tian

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026 · 1-18

DOI: 10.1109/tpami.2026.3748834 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026. Verfasst von Junjie Wang und 6 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Dense visual perception tasks have been constrained by their reliance on predefined categories, limiting their applicability in real-world scenarios where visual concepts are unbounded. While Vision-Language Models (VLMs) like CLIP have shown promise in open-vocabulary tasks, their direct application to dense perception often leads to suboptimal performance due to limitations in local feature representation. In this work, we present our observation that CLIP's image tokens struggle to effectively aggregate information from spatially or semantically related regions, resulting in features that lack local discriminability and spatial consistency. To address this issue, we propose DeCLIP, a novel framework that enhances CLIP by decoupling the self-attention module to obtain "content" and "context" features respectively. The context features are enhanced by jointly distilling semantic correlations from Vision Foundation Models (VFMs) and object integrity cues from diffusion models, thereby enhancing spatial consistency. In parallel, the content features are aligned with image crop representations and constrained by region correlations from VFMs to improve local discriminability. Extensive experiments demonstrate that DeCLIP establishes a solid foundation for open-vocabulary dense perception, consistently achieving state-of-the-art performance across a broad spectrum of tasks, including 2D detection and segmentation, 3D instance segmentation, video instance segmentation, and 6D object pose estimation.

Zitieren

APA

Wang, J., Chen, K., Li, Y., Chen, B., Zhao, H., Qi, X., & Tian, Z. (2026). Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1-18. https://doi.org/10.1109/tpami.2026.3748834

Vancouver

Wang J, Chen K, Li Y, Chen B, Zhao H, Qi X, et al. Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026:1-18. doi:10.1109/tpami.2026.3748834

BibTeX
@article{Wang2026,
  author = {Wang, Junjie and Chen, Keyu and Li, Yulin and Chen, Bin and Zhao, Hengshuang and Qi, Xiaojuan and Tian, Zhuotao},
  title = {Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  pages = {1--18},
  doi = {10.1109/tpami.2026.3748834}
}
Kostenlos starten →