EUVIMEDEUVIMED entdecken
Crossref

Exploring Rich Semantics for Open-Set Action Recognition

Yufan Hu, Junyu Gao, Jianfeng Dong, Bin Fan, Hongmin Liu

IEEE Transactions on Multimedia · 2024 · 26 · 5410-5421

DOI: 10.1109/tmm.2023.3333206 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Multimedia, 2024. Verfasst von Yufan Hu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 60-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Beijing Natural Science Foundation, China Postdoctoral Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Open-set action recognition (OSAR) aims to learn a recognition framework capable of both classifying known classes and identifying unknown actions in open-set scenarios. Existing OSAR methods typically reside in a data-driven paradigm, which ignore the rich semantics in both known and unknown categories. In fact, we humans have the capability of leveraging the captured semantic information, i.e., knowledge and experience, to incisively distinguish samples from known and unknown classes. Motivated by this observation, in this paper, we propose a Unified Semantic Exploration (USE) framework for recognizing actions in open-set scenarios. Specifically, we explore the explicit knowledge semantics by simulating the unknown classes with knowledge-guided virtual classes based on an external knowledge graph, which enables the model to simulate open-set perception during model training. Besides, we propose to learn the implicit data semantics by transferring the knowledge structure of action categories to the visual prototype space for semantic structure preservation. Extensive experiments on several action recognition benchmarks validate the effectiveness of our proposed method.

Förderung

  • National Natural Science Foundation of China
  • Beijing Natural Science Foundation
  • China Postdoctoral Science Foundation
  • Fundamental Research Funds for the Central Universities
  • State Key Laboratory of Multimodal Artificial Intelligence Systems

Zitieren

APA

Hu, Y., Gao, J., Dong, J., Fan, B., & Liu, H. (2024). Exploring Rich Semantics for Open-Set Action Recognition. IEEE Transactions on Multimedia, 26, 5410-5421. https://doi.org/10.1109/tmm.2023.3333206

Vancouver

Hu Y, Gao J, Dong J, Fan B, Liu H. Exploring Rich Semantics for Open-Set Action Recognition. IEEE Transactions on Multimedia. 2024;26:5410-5421. doi:10.1109/tmm.2023.3333206

BibTeX
@article{Hu2024,
  author = {Hu, Yufan and Gao, Junyu and Dong, Jianfeng and Fan, Bin and Liu, Hongmin},
  title = {Exploring Rich Semantics for Open-Set Action Recognition},
  journal = {IEEE Transactions on Multimedia},
  year = {2024},
  volume = {26},
  pages = {5410--5421},
  doi = {10.1109/tmm.2023.3333206}
}
Kostenlos starten →