GCN-Transformer: Graph Convolutional Network and Transformer for Multi-Person Pose Forecasting Using Sensor-Based Motion Data
Sensors · 15.05.2025 · 25 (10) · 3136
DOI: 10.3390/s25103136 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 15.05.2025. Verfasst von Romeo Šajina und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Informatics and Digital Technologies, University of Rijeka, 51000 Rijeka, Croatia. Hauptthemen laut MEDLINE: Neural Networks, Computer, Posture. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Croatian Science Foundation, University of Rijeka.
Abstract (Originalsprache)
Multi-person pose forecasting involves predicting the future body poses of multiple individuals over time, involving complex movement dynamics and interaction dependencies. Its relevance spans various fields, including computer vision, robotics, human–computer interaction, and surveillance. This task is particularly important in sensor-driven applications, where motion capture systems, including vision-based sensors and IMUs, provide crucial data for analyzing human movement. This paper introduces GCN-Transformer, a novel model for multi-person pose forecasting that leverages the integration of Graph Convolutional Network and Transformer architectures. We integrated novel loss terms during the training phase to enable the model to learn both interaction dependencies and the trajectories of multiple joints simultaneously. Additionally, we propose a novel pose forecasting evaluation metric called Final Joint Position and Trajectory Error (FJPTE), which assesses both local movement dynamics and global movement errors by considering the final position and the trajectory leading up to it, providing a more comprehensive assessment of movement dynamics. Our model uniquely integrates scene-level graph-based encoding and personalized attention-based decoding, introducing a novel architecture for multi-person pose forecasting that achieves state-of-the-art results across four datasets. The model is trained and evaluated on the CMU-Mocap, MuPoTS-3D, SoMoF Benchmark, and ExPI datasets, which are collected using sensor-based motion capture systems, ensuring its applicability in real-world scenarios. Comprehensive evaluations on the CMU-Mocap, MuPoTS-3D, SoMoF Benchmark, and ExPI datasets demonstrate that the proposed GCN-Transformer model consistently outperforms existing state-of-the-art (SOTA) models according to the VIM and MPJPE metrics. Specifically, based on the MPJPE metric, GCN-Transformer shows a 4.7% improvement over the closest SOTA model on CMU-Mocap, 4.3% improvement over the closest SOTA model on MuPoTS-3D, 5% improvement over the closest SOTA model on the SoMoF Benchmark, and a 2.6% improvement over the closest SOTA model on the ExPI dataset. Unlike other models with performances that fluctuate across datasets, GCN-Transformer performs consistently, proving its robustness in multi-person pose forecasting and providing an excellent foundation for the application of GCN-Transformer in different domains.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Informatics and Digital Technologies, University of Rijeka, 51000 Rijeka, Croatia.
- Faculty of Informatics, Juraj Dobrila University of Pula, 52100 Pula, Croatia.
- Centre for Artificial Intelligence, University of Rijeka, 51000 Rijeka, Croatia.
ORCID-Profile: Romeo Šajina, Goran Oreški, Marina Ivašić-Kos
Förderung
- Croatian Science Foundation
- University of Rijeka
Zitieren
APA
Šajina, R., Oreški, G., & Ivašić-Kos, M. (2025). GCN-Transformer: Graph Convolutional Network and Transformer for Multi-Person Pose Forecasting Using Sensor-Based Motion Data. Sensors, 25(10), 3136. https://doi.org/10.3390/s25103136
Vancouver
Šajina R, Oreški G, Ivašić-Kos M. GCN-Transformer: Graph Convolutional Network and Transformer for Multi-Person Pose Forecasting Using Sensor-Based Motion Data. Sensors. 2025;25(10):3136. doi:10.3390/s25103136
BibTeX
@article{ajina2025,
author = {Šajina, Romeo and Oreški, Goran and Ivašić-Kos, Marina},
title = {GCN-Transformer: Graph Convolutional Network and Transformer for Multi-Person Pose Forecasting Using Sensor-Based Motion Data},
journal = {Sensors},
year = {2025},
volume = {25},
number = {10},
pages = {3136},
doi = {10.3390/s25103136}
}
EUVIMED