Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation
IEEE Transactions on Neural Networks and Learning Systems · 2025-09 · 36 (9) · 17462-17476
DOI: 10.1109/tnnls.2025.3563025 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Neural Networks and Learning Systems, 2025-09. Verfasst von Haoyu Ji und 5 weiteren Autor:innen. Hauptthemen laut MEDLINE: Biomechanical Phenomena, Movement. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Shenzhen Science and Technology Program.
Abstract (Originalsprache)
The skeleton-based temporal action segmentation (STAS) aims to densely segment and classify human actions within lengthy untrimmed skeletal motion sequences. Current methods primarily rely on graph convolutional networks (GCNs) for intraframe spatial modeling and temporal convolutional networks (TCNs) for interframe temporal modeling to discern motion patterns. However, these approaches often overlook the distinctive nature of essential action elements across various actions, including engaged core body parts and key subactions. This oversight limits the ability to distinguish different actions within a given sequence. To address these limitations, the snippet-aware Transformer with multiple action element (ME-ST) is proposed to enhance the discrimination and segmentation among actions, which leverages intrasnippet attention along joints and sequences to identify core joints and key subactions at different scales. Specifically, in terms of the spatial domain, the intrasnippet cross-joint attention (CJA) module divides the sequence into distinct snippets and computes attention to establish intricate joint semantic relationships, emphasizing the identification of core motion joints. In terms of the temporal domain, in the encoder, the intrasnippet cross-frame attention (CFA) module segments the sequence in a blockwise expansion manner and establishes interframe relationships to highlight the most discriminative frames. In the decoder, clip-level representations at various temporal scales are initially generated through an hourglass-like sampling process, followed by the intrasnippet cross-scale attention (CSA) module to integrate the key clip information across different time scales. The performance evaluation on five public datasets demonstrates that ME-ST achieves state-of-the-art (SOTA) performance.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Shenzhen Science and Technology Program
- Science and Technology Development Fund (FDCT), Macau, SAR
Zitieren
APA
Ji, H., Chen, B., Huang, W., Ren, W., Wang, Z., & Liu, H. (2025). Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation. IEEE Transactions on Neural Networks and Learning Systems, 36(9), 17462-17476. https://doi.org/10.1109/tnnls.2025.3563025
Vancouver
Ji H, Chen B, Huang W, Ren W, Wang Z, Liu H. Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation. IEEE Transactions on Neural Networks and Learning Systems. 2025;36(9):17462-17476. doi:10.1109/tnnls.2025.3563025
BibTeX
@article{Ji2025,
author = {Ji, Haoyu and Chen, Bowen and Huang, Wenze and Ren, Weihong and Wang, Zhiyong and Liu, Honghai},
title = {Snippet-Aware Transformer With Multiple Action Elements for Skeleton-Based Action Segmentation},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
year = {2025},
volume = {36},
number = {9},
pages = {17462--17476},
doi = {10.1109/tnnls.2025.3563025}
}
EUVIMED