EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Towards Open Vocabulary Learning: A Survey

Jianzong Wu, Xiangtai Li, Shilin Xu, Haobo Yuan, Henghui Ding, Yibo Yang, Xia Li, Jiangning Zhang, Yunhai Tong, Xudong Jiang, Bernard Ghanem, Dacheng Tao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-07 · 46 (7) · 5092-5113

DOI: 10.1109/tpami.2024.3361862 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-07. Verfasst von Jianzong Wu und 11 weiteren Autor:innen. Die Arbeit wurde bisher 174-mal zitiert (Crossref) und nennt 323 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Key Research and Development Program of China, interdisciplinary doctoral, Personalized Health and Related Technologies.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the field of visual scene understanding, deep neural networks have made impressive advancements in various core tasks like segmentation, tracking, and detection. However, most approaches operate on the close-set assumption, meaning that the model can only identify pre-defined categories that are present in the training set. Recently, open vocabulary settings were proposed due to the rapid progress of vision language pre-training. These new approaches seek to locate and recognize categories beyond the annotated label space. The open vocabulary approach is more general, practical, and effective than weakly supervised and zero-shot settings. This paper thoroughly reviews open vocabulary learning, summarizing and analyzing recent developments in the field. In particular, we begin by juxtaposing open vocabulary learning with analogous concepts such as zero-shot learning, open-set recognition, and out-of-distribution detection. Subsequently, we examine several pertinent tasks within the realms of segmentation and detection, encompassing long-tail problems, few-shot, and zero-shot settings. As a foundation for our method survey, we first elucidate the fundamental principles of detection and segmentation in close-set scenarios. Next, we examine various contexts where open vocabulary learning is employed, pinpointing recurring design elements and central themes. This is followed by a comparative analysis of recent detection and segmentation methodologies in commonly used datasets and benchmarks. Our review culminates with a synthesis of insights, challenges, and discourse on prospective research trajectories. To our knowledge, this constitutes the inaugural exhaustive literature review on open vocabulary learning.

Förderung

  • National Key Research and Development Program of China
  • interdisciplinary doctoral
  • Personalized Health and Related Technologies

Zitieren

APA

Wu, J., Li, X., Xu, S., Yuan, H., Ding, H., Yang, Y., Li, X., Zhang, J., Tong, Y., Jiang, X., Ghanem, B., & Tao, D. (2024). Towards Open Vocabulary Learning: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(7), 5092-5113. https://doi.org/10.1109/tpami.2024.3361862

Vancouver

Wu J, Li X, Xu S, Yuan H, Ding H, Yang Y, et al. Towards Open Vocabulary Learning: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(7):5092-5113. doi:10.1109/tpami.2024.3361862

BibTeX
@article{Wu2024,
  author = {Wu, Jianzong and Li, Xiangtai and Xu, Shilin and Yuan, Haobo and Ding, Henghui and Yang, Yibo and Li, Xia and Zhang, Jiangning and Tong, Yunhai and Jiang, Xudong and Ghanem, Bernard and Tao, Dacheng},
  title = {Towards Open Vocabulary Learning: A Survey},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {7},
  pages = {5092--5113},
  doi = {10.1109/tpami.2024.3361862}
}
Kostenlos starten →