EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Category-Aware Saliency Enhance Learning Based on CLIP for Weakly Supervised Salient Object Detection

Yunde Zhang, Zhili Zhang, Tianshan Liu, Jun Kong

Neural Processing Letters · 16.02.2024 · 56 (2)

DOI: 10.1007/s11063-024-11530-2 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neural Processing Letters, online erschienen am 16.02.2024. Verfasst von Yunde Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Scientific and Technological Aid Program of Xinjiang, 111 Projects under Grant, the National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Weakly supervised salient object detection (SOD) using image-level category labels has been proposed to reduce the annotation cost of pixel-level labels. However, existing methods mostly train a classification network to generate a class activation map, which suffers from coarse localization and difficult pseudo-label updating. To address these issues, we propose a novel Category-aware Saliency Enhance Learning (CSEL) method based on contrastive vision-language pre-training (CLIP), which can perform image-text classification and pseudo-label updating simultaneously. Our proposed method transforms image-text classification into pixel-text matching and generates a category-aware saliency map, which is evaluated by the classification accuracy. Moreover, CSEL assesses the quality of the category-aware saliency map and the pseudo saliency map, and uses the quality confidence scores as weights to update the pseudo labels. The two maps mutually enhance each other to guide the pseudo saliency map in the correct direction. Our SOD network can be trained jointly under the supervision of the updated pseudo saliency maps. We test our model on various well-known RGB-D and RGB SOD datasets. Our model achieves an S-measure of 87.6 $$\%$$ % on the RGB-D NLPR dataset and 84.3 $$\%$$ % on the RGB ECSSD dataset. Additionally, we obtain satisfactory performance on the weakly supervised E-measure, F-measure, and mean absolute error metrics for other datasets. These results demonstrate the effectiveness of our model.

Förderung

  • Scientific and Technological Aid Program of Xinjiang
  • 111 Projects under Grant
  • the National Natural Science Foundation of China

Zitieren

APA

Zhang, Y., Zhang, Z., Liu, T., & Kong, J. (2024). Category-Aware Saliency Enhance Learning Based on CLIP for Weakly Supervised Salient Object Detection. Neural Processing Letters, 56(2). https://doi.org/10.1007/s11063-024-11530-2

Vancouver

Zhang Y, Zhang Z, Liu T, Kong J. Category-Aware Saliency Enhance Learning Based on CLIP for Weakly Supervised Salient Object Detection. Neural Processing Letters. 2024;56(2). doi:10.1007/s11063-024-11530-2

BibTeX
@article{Zhang2024,
  author = {Zhang, Yunde and Zhang, Zhili and Liu, Tianshan and Kong, Jun},
  title = {Category-Aware Saliency Enhance Learning Based on CLIP for Weakly Supervised Salient Object Detection},
  journal = {Neural Processing Letters},
  year = {2024},
  volume = {56},
  number = {2},
  doi = {10.1007/s11063-024-11530-2}
}
Kostenlos starten →