Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination
Sensors · 2024
DOI: 10.3390/s24041189 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Lin Yuan und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China. Hauptthemen laut MEDLINE: Movement, Data Analysis. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
This paper introduces ‘SkeletonCLIP++’, an extension of our prior work in human action recognition, emphasizing the use of semantic information beyond traditional label-based methods. The innovation, ‘Weighted Frame Integration’ (WFI), shifts video feature computation from averaging to a weighted frame approach, enabling a more nuanced representation of human movements in line with semantic relevance. Another key development, ‘Contrastive Sample Identification’ (CSI), introduces a novel discriminative task within the model. This task involves identifying the most similar negative sample among positive ones, enhancing the model’s ability to distinguish between closely related actions. Incorporating the ‘BERT Text Encoder Integration’ (BTEI) leverages the pre-trained BERT model as our text encoder to refine the model’s performance. Empirical evaluations on HMDB-51, UCF-101, and NTU RGB+D 60 datasets illustrate positive improvements, especially in smaller datasets. ‘SkeletonCLIP++’ thus offers a refined approach to human action recognition, ensuring semantic integrity and detailed differentiation in video data analysis.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China.
ORCID-Profile: Qiang Wang, Leiyang Xu
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Yuan, L., He, Z., Wang, Q., & Xu, L. (2024). Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination. Sensors. https://doi.org/10.3390/s24041189
Vancouver
Yuan L, He Z, Wang Q, Xu L. Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination. Sensors. 2024. doi:10.3390/s24041189
BibTeX
@article{Yuan2024,
author = {Yuan, Lin and He, Zhen and Wang, Qiang and Xu, Leiyang},
title = {Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24041189}
}
EUVIMED