Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment
IEEE Transactions on Image Processing · 2025 · 34 · 4602-4617
DOI: 10.1109/tip.2025.3586487 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2025. Verfasst von Kai Zhou und 5 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 90 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Guangdong Basic and Applied Basic Research Foundation.
Abstract (Originalsprache)
Zero-shot skeleton-based action recognition aims to classify unseen skeleton-based human actions without prior exposure to such categories during training. This task is extremely challenging due to the difficulty in generalizing from known to unknown actions. Previous studies typically use two-stage training: pre-training skeleton encoders on seen action categories using cross-entropy loss and then aligning pre-extracted skeleton and text features, enabling knowledge transfer to unseen classes through skeleton-text alignment and language models' generalization. However, their efficacy is hindered by 1) insufficient discrimination for skeleton features, as the fixed skeleton encoder fails to capture necessary alignment information for effective skeleton-text alignment; 2) the neglect of alignment bias between skeleton and unseen text features during testing. To this end, we propose a prototype-guided feature alignment paradigm for zero-shot skeleton-based action recognition, termed PGFA. Specifically, we develop an end-to-end cross-modal contrastive training framework to improve skeleton-text alignment, ensuring sufficient discrimination for skeleton features. Additionally, we introduce a prototype-guided text feature alignment strategy to mitigate the adverse impact of the distribution discrepancy during testing. We provide a theoretical analysis to support our prototype-guided text feature alignment strategy and empirically evaluate our overall PGFA on three well-known datasets. Compared with the top competitor SMIE method, our PGFA achieves absolute accuracy improvements of 22.96%, 12.53%, and 18.54% on the NTU-60, NTU-120, and PKU-MMD datasets, respectively.
Förderung
- National Natural Science Foundation of China
- Guangdong Basic and Applied Basic Research Foundation
Zitieren
APA
Zhou, K., Zhang, S., You, Z., Hu, J., Tan, M., & Liu, F. (2025). Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment. IEEE Transactions on Image Processing, 34, 4602-4617. https://doi.org/10.1109/tip.2025.3586487
Vancouver
Zhou K, Zhang S, You Z, Hu J, Tan M, Liu F. Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment. IEEE Transactions on Image Processing. 2025;34:4602-4617. doi:10.1109/tip.2025.3586487
BibTeX
@article{Zhou2025,
author = {Zhou, Kai and Zhang, Shuhai and You, Zeng and Hu, Jinwu and Tan, Mingkui and Liu, Fei},
title = {Zero-Shot Skeleton-Based Action Recognition With Prototype-Guided Feature Alignment},
journal = {IEEE Transactions on Image Processing},
year = {2025},
volume = {34},
pages = {4602--4617},
doi = {10.1109/tip.2025.3586487}
}
EUVIMED