Weakly Supervised Open-Vocabulary Object Detection
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (4) · 3404-3412
DOI: 10.1609/aaai.v38i4.28127 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Jianghang Lin und 5 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref).
Abstract (Originalsprache)
Despite weakly supervised object detection (WSOD) being a promising step toward evading strong instance-level annotations, its capability is confined to closed-set categories within a single training dataset. In this paper, we propose a novel weakly supervised open-vocabulary object detection framework, namely WSOVOD, to extend traditional WSOD to detect novel concepts and utilize diverse datasets with only image-level annotations. To achieve this, we explore three vital strategies, including dataset-level feature adaptation, image-level salient object localization, and region-level vision-language alignment. First, we perform data-aware feature extraction to produce an input-conditional coefficient, which is leveraged into dataset attribute prototypes to identify dataset bias and help achieve cross-dataset generalization. Second, a customized location-oriented weakly supervised region proposal network is proposed to utilize high-level semantic layouts from the category-agnostic segment anything model to distinguish object boundaries. Lastly, we introduce a proposal-concept synchronized multiple-instance network, i.e., object mining and refinement with visual-semantic alignment, to discover objects matched to the text embeddings of concepts. Extensive experiments on Pascal VOC and MS COCO demonstrate that the proposed WSOVOD achieves new state-of-the-art compared with previous WSOD methods in both close-set object localization and detection tasks. Meanwhile, WSOVOD enables cross-dataset and open-vocabulary learning to achieve on-par or even better performance than well-established fully-supervised open-vocabulary object detection (FSOVOD).
Zitieren
APA
Lin, J., Shen, Y., Wang, B., Lin, S., Li, K., & Cao, L. (2024). Weakly Supervised Open-Vocabulary Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4), 3404-3412. https://doi.org/10.1609/aaai.v38i4.28127
Vancouver
Lin J, Shen Y, Wang B, Lin S, Li K, Cao L. Weakly Supervised Open-Vocabulary Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(4):3404-3412. doi:10.1609/aaai.v38i4.28127
BibTeX
@article{Lin2024,
author = {Lin, Jianghang and Shen, Yunhang and Wang, Bingquan and Lin, Shaohui and Li, Ke and Cao, Liujuan},
title = {Weakly Supervised Open-Vocabulary Object Detection},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {4},
pages = {3404--3412},
doi = {10.1609/aaai.v38i4.28127}
}
EUVIMED