A Visual Representation-Guided Framework With Global Affinity for Weakly Supervised Salient Object Detection
IEEE Transactions on Circuits and Systems for Video Technology · 2024-01 · 34 (1) · 248-259
DOI: 10.1109/tcsvt.2023.3284076 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-01. Verfasst von Binwei Xu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 74 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Zhejiang Provincial Natural Science Foundation of China.
Abstract (Originalsprache)
Fully supervised salient object detection (SOD) methods have made considerable progress in performance, yet these models rely heavily on expensive pixel-wise labels. Recently, to achieve a trade-off between labeling burden and performance, scribble-based SOD methods have attracted increasing attention. Previous scribble-based models directly implement the SOD task only based on SOD training data with limited information, it is extremely difficult for them to understand the image and further achieve a superior SOD task. In this paper, we propose a simple yet effective framework guided by general visual representations with rich contextual semantic knowledge for scribble-based SOD. These general visual representations are generated by self-supervised learning based on large-scale unlabeled datasets. Our framework consists of a task-related encoder, a general visual module, and an information integration module to efficiently combine the general visual representations with task-related features to perform the SOD task based on understanding the contextual connections of images. Meanwhile, we propose a novel global semantic affinity loss to guide the model to perceive the global structure of the salient objects. Experimental results on five public benchmark datasets demonstrate that our method, which only utilizes scribble annotations without introducing any extra label, outperforms the state-of-theart weakly supervised SOD methods. Specifically, it outperforms the previous best scribble-based method on all datasets with an average gain of 5.5% for max f-measure, 5.8% for mean f-measure, 24% for MAE, and 3.1% for E-measure. Moreover, our method achieves comparable or even superior performance to the state-of-the-art fully supervised models.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Zhejiang Provincial Natural Science Foundation of China
Zitieren
APA
Xu, B., Liang, H., Gong, W., Liang, R., & Chen, P. (2024). A Visual Representation-Guided Framework With Global Affinity for Weakly Supervised Salient Object Detection. IEEE Transactions on Circuits and Systems for Video Technology, 34(1), 248-259. https://doi.org/10.1109/tcsvt.2023.3284076
Vancouver
Xu B, Liang H, Gong W, Liang R, Chen P. A Visual Representation-Guided Framework With Global Affinity for Weakly Supervised Salient Object Detection. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(1):248-259. doi:10.1109/tcsvt.2023.3284076
BibTeX
@article{Xu2024,
author = {Xu, Binwei and Liang, Haoran and Gong, Weihua and Liang, Ronghua and Chen, Peng},
title = {A Visual Representation-Guided Framework With Global Affinity for Weakly Supervised Salient Object Detection},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {1},
pages = {248--259},
doi = {10.1109/tcsvt.2023.3284076}
}
EUVIMED