EUVIMEDEUVIMED entdecken
Crossref

Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation

Haoyu Ji, Bowen Chen, Wenze Huang, Weihong Ren, Zhiyong Wang, Honghai Liu

IEEE Transactions on Neural Networks and Learning Systems · 2025-09 · 36 (9) · 17462-17476

DOI: 10.1109/tnnls.2025.3563025 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Neural Networks and Learning Systems, 2025-09. Verfasst von Haoyu Ji und 5 weiteren Autor:innen. Hauptthemen laut MEDLINE: Biomechanical Phenomena, Movement. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Shenzhen Science and Technology Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The skeleton-based temporal action segmentation (STAS) aims to densely segment and classify human actions within lengthy untrimmed skeletal motion sequences. Current methods primarily rely on graph convolutional networks (GCNs) for intraframe spatial modeling and temporal convolutional networks (TCNs) for interframe temporal modeling to discern motion patterns. However, these approaches often overlook the distinctive nature of essential action elements across various actions, including engaged core body parts and key subactions. This oversight limits the ability to distinguish different actions within a given sequence. To address these limitations, the snippet-aware Transformer with multiple action element (ME-ST) is proposed to enhance the discrimination and segmentation among actions, which leverages intrasnippet attention along joints and sequences to identify core joints and key subactions at different scales. Specifically, in terms of the spatial domain, the intrasnippet cross-joint attention (CJA) module divides the sequence into distinct snippets and computes attention to establish intricate joint semantic relationships, emphasizing the identification of core motion joints. In terms of the temporal domain, in the encoder, the intrasnippet cross-frame attention (CFA) module segments the sequence in a blockwise expansion manner and establishes interframe relationships to highlight the most discriminative frames. In the decoder, clip-level representations at various temporal scales are initially generated through an hourglass-like sampling process, followed by the intrasnippet cross-scale attention (CSA) module to integrate the key clip information across different time scales. The performance evaluation on five public datasets demonstrates that ME-ST achieves state-of-the-art (SOTA) performance.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Biomechanical PhenomenaMovementHumansSkeletonNeural Networks, Computer

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Shenzhen Science and Technology Program
  • Science and Technology Development Fund (FDCT), Macau, SAR

Zitieren

APA

Ji, H., Chen, B., Huang, W., Ren, W., Wang, Z., & Liu, H. (2025). Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation. IEEE Transactions on Neural Networks and Learning Systems, 36(9), 17462-17476. https://doi.org/10.1109/tnnls.2025.3563025

Vancouver

Ji H, Chen B, Huang W, Ren W, Wang Z, Liu H. Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation. IEEE Transactions on Neural Networks and Learning Systems. 2025;36(9):17462-17476. doi:10.1109/tnnls.2025.3563025

BibTeX
@article{Ji2025,
  author = {Ji, Haoyu and Chen, Bowen and Huang, Wenze and Ren, Weihong and Wang, Zhiyong and Liu, Honghai},
  title = {Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation},
  journal = {IEEE Transactions on Neural Networks and Learning Systems},
  year = {2025},
  volume = {36},
  number = {9},
  pages = {17462--17476},
  doi = {10.1109/tnnls.2025.3563025}
}
Kostenlos starten →