EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination

Lin Yuan, Zhen He, Qiang Wang iD, Leiyang Xu iD

Sensors · 2024

DOI: 10.3390/s24041189 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Lin Yuan und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China. Hauptthemen laut MEDLINE: Movement, Data Analysis. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper introduces ‘SkeletonCLIP++’, an extension of our prior work in human action recognition, emphasizing the use of semantic information beyond traditional label-based methods. The innovation, ‘Weighted Frame Integration’ (WFI), shifts video feature computation from averaging to a weighted frame approach, enabling a more nuanced representation of human movements in line with semantic relevance. Another key development, ‘Contrastive Sample Identification’ (CSI), introduces a novel discriminative task within the model. This task involves identifying the most similar negative sample among positive ones, enhancing the model’s ability to distinguish between closely related actions. Incorporating the ‘BERT Text Encoder Integration’ (BTEI) leverages the pre-trained BERT model as our text encoder to refine the model’s performance. Empirical evaluations on HMDB-51, UCF-101, and NTU RGB+D 60 datasets illustrate positive improvements, especially in smaller datasets. ‘SkeletonCLIP++’ thus offers a refined approach to human action recognition, ensuring semantic integrity and detailed differentiation in video data analysis.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

MovementData AnalysisHumansMotionRecognition, PsychologySemantics

Stichwörter der Autor:innen

action recognitioncontrastive learningmulti-modal fusion

Autor:innen und Institutionen

  • Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China.

ORCID-Profile: Qiang Wang, Leiyang Xu

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Yuan, L., He, Z., Wang, Q., & Xu, L. (2024). Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination. Sensors. https://doi.org/10.3390/s24041189

Vancouver

Yuan L, He Z, Wang Q, Xu L. Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination. Sensors. 2024. doi:10.3390/s24041189

BibTeX
@article{Yuan2024,
  author = {Yuan, Lin and He, Zhen and Wang, Qiang and Xu, Leiyang},
  title = {Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24041189}
}
Kostenlos starten →