EUVIMEDEUVIMED entdecken
Crossref

Navigating Open Set Scenarios for Skeleton-Based Action Recognition

Kunyu Peng, Cheng Yin, Junwei Zheng, Ruiping Liu, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4487-4496

DOI: 10.1609/aaai.v38i5.28247 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Kunyu Peng und 9 weiteren Autor:innen. Die Arbeit wurde bisher 31-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In real-world scenarios, human actions often fall outside the distribution of training data, making it crucial for models to recognize known actions and reject unknown ones. However, using pure skeleton data in such open-set conditions poses challenges due to the lack of visual background cues and the distinct sparse structure of body pose sequences. In this paper, we tackle the unexplored Open-Set Skeleton-based Action Recognition (OS-SAR) task and formalize the benchmark on three skeleton-based datasets. We assess the performance of seven established open-set approaches on our task and identify their limits and critical generalization issues when dealing with skeleton information.To address these challenges, we propose a distance-based cross-modality ensemble method that leverages the cross-modal alignment of skeleton joints, bones, and velocities to achieve superior open-set recognition performance. We refer to the key idea as CrossMax - an approach that utilizes a novel cross-modality mean max discrepancy suppression mechanism to align latent spaces during training and a cross-modality distance-based logits refinement method during testing. CrossMax outperforms existing approaches and consistently yields state-of-the-art results across all datasets and backbones. We will release the benchmark, code, and models to the community.

Zitieren

APA

Peng, K., Yin, C., Zheng, J., Liu, R., Schneider, D., Zhang, J., Yang, K., Sarfraz, M. S., Stiefelhagen, R., & Roitberg, A. (2024). Navigating Open Set Scenarios for Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4487-4496. https://doi.org/10.1609/aaai.v38i5.28247

Vancouver

Peng K, Yin C, Zheng J, Liu R, Schneider D, Zhang J, et al. Navigating Open Set Scenarios for Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4487-4496. doi:10.1609/aaai.v38i5.28247

BibTeX
@article{Peng2024,
  author = {Peng, Kunyu and Yin, Cheng and Zheng, Junwei and Liu, Ruiping and Schneider, David and Zhang, Jiaming and Yang, Kailun and Sarfraz, M. Saquib and Stiefelhagen, Rainer and Roitberg, Alina},
  title = {Navigating Open Set Scenarios for Skeleton-Based Action Recognition},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {5},
  pages = {4487--4496},
  doi = {10.1609/aaai.v38i5.28247}
}
Kostenlos starten →