Skeleton Reconstruction Using Generative Adversarial Networks for Human Activity Recognition Under Occlusion
Sensors · 2025
DOI: 10.3390/s25051567 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Ioannis Vernikos und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Informatics and Telecommunications, University of Thessaly, 35100 Lamia, Greece. Hauptthemen laut MEDLINE: Generative Adversarial Networks, Human Activities, Image Processing, Computer-Assisted, Pattern Recognition, Automated. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 67 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Recognizing human activities from motion data is a complex task in computer vision, involving the recognition of human behaviors from sequences of 3D motion data. These activities encompass successive body part movements, interactions with objects, or group dynamics. Camera-based recognition methods are cost-effective and perform well under controlled conditions but face challenges in real-world scenarios due to factors such as viewpoint changes, illumination variations, and occlusion. The latter is the most significant challenge in real-world recognition; partial occlusion impacts recognition accuracy to varying degrees depending on the activity and the occluded body parts while complete occlusion can render activity recognition impossible. In this paper, we propose a novel approach for human activity recognition in the presence of partial occlusion, which may be applied in cases wherein up to two body parts are occluded. The proposed approach works under the assumptions that (a) human motion is modeled using a set of 3D skeletal joints, and (b) the same body parts remain occluded throughout the whole activity. Contrary to previous research, in this work, we address this problem using a Generative Adversarial Network (GAN). Specifically, we train a Convolutional Recurrent Neural Network (CRNN), whose goal is to serve as the generator of the GAN. Its aim is to complete the missing parts of the skeleton due to occlusion. Specifically, the input to this CRNN consists of raw 3D skeleton joint positions, upon the removal of joints corresponding to occluded parts. The output of the CRNN is a reconstructed skeleton. For the discriminator of the GAN, we use a simple long short-term memory (LSTM) network. We evaluate the proposed approach using publicly available datasets in a series of occlusion scenarios. We demonstrate that in all scenarios, the occlusion of certain body parts causes a significant decline in performance, although in some cases, the reconstruction process leads to almost perfect recognition. Nonetheless, in almost every circumstance, the herein proposed approach exhibits superior performance compared to previous works, which varies between 2.2% and 37.5%, depending on the dataset used and the occlusion case.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Informatics and Telecommunications, University of Thessaly, 35100 Lamia, Greece.
ORCID-Profile: Ioannis Vernikos
Zitieren
APA
Vernikos, I., & Spyrou, E. (2025). Skeleton Reconstruction Using Generative Adversarial Networks for Human Activity Recognition Under Occlusion. Sensors. https://doi.org/10.3390/s25051567
Vancouver
Vernikos I, Spyrou E. Skeleton Reconstruction Using Generative Adversarial Networks for Human Activity Recognition Under Occlusion. Sensors. 2025. doi:10.3390/s25051567
BibTeX
@article{Vernikos2025,
author = {Vernikos, Ioannis and Spyrou, Evaggelos},
title = {Skeleton Reconstruction Using Generative Adversarial Networks for Human Activity Recognition Under Occlusion},
journal = {Sensors},
year = {2025},
doi = {10.3390/s25051567}
}
EUVIMED