EUVIMEDEUVIMED entdecken
Crossref

Multi-Modal Prompts With Primitives Enhancement for Compositional Zero-Shot Learning

Yutang Jin, Shiming Chen, Tianle Tong, Weiping Ding, Yisong Wang

IEEE Transactions on Circuits and Systems for Video Technology · 2025-12 · 35 (12) · 11754-11769

DOI: 10.1109/tcsvt.2025.3577246 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-12. Verfasst von Yutang Jin und 4 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 75 Literaturangaben. Gefördert von: National Key Research and Development Plan of China, National Natural Science Foundation of China, Guizhou Science Support Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Compositional zero-shot learning (CZSL) aims to recognize novel compositions of known attributes and objects without requiring additional training data. Recent CZSL methods based on vision-language models(e.g., CLIP) suffer from relying solely on text prompts and neglecting the crucial primitive features within compositions, which limits generalization to unseen compositions. To overcome these limitations, we propose a Multi-modal Prompt and Primitives Enhancement method, termed MPPE, which incorporates two key aspects. First, MPPE introduces both text and visual prompts. The text prompts consist of the composition and its corresponding attribute and object prompts, while the visual prompts leverage image masks generated by the segment anything model (SAM). These masks are integrated via an additional Alpha branch to strengthen the CLIP visual encoder to focus on regions of interest within the image. Second, we design a primitives enhancement (PE) module based on cross-attention, which refines attribute and object features obtained from the CLIP text encoder, thereby enriching the representation of novel composition features. Extensive experiments demonstrate the effectiveness of our approach, achieving state-of-the-art performance on three widely-used CZSL benchmarks in both closed-world and open-world CZSL scenarios. Codes are available at https://github.com/YtJin-git/MPPE

Förderung

  • National Key Research and Development Plan of China
  • National Natural Science Foundation of China
  • Guizhou Science Support Project

Zitieren

APA

Jin, Y., Chen, S., Tong, T., Ding, W., & Wang, Y. (2025). Multi-Modal Prompts With Primitives Enhancement for Compositional Zero-Shot Learning. IEEE Transactions on Circuits and Systems for Video Technology, 35(12), 11754-11769. https://doi.org/10.1109/tcsvt.2025.3577246

Vancouver

Jin Y, Chen S, Tong T, Ding W, Wang Y. Multi-Modal Prompts With Primitives Enhancement for Compositional Zero-Shot Learning. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(12):11754-11769. doi:10.1109/tcsvt.2025.3577246

BibTeX
@article{Jin2025,
  author = {Jin, Yutang and Chen, Shiming and Tong, Tianle and Ding, Weiping and Wang, Yisong},
  title = {Multi-Modal Prompts With Primitives Enhancement for Compositional Zero-Shot Learning},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2025},
  volume = {35},
  number = {12},
  pages = {11754--11769},
  doi = {10.1109/tcsvt.2025.3577246}
}
Kostenlos starten →