EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation

Sajid Ur Rehman iD, Aman Ullah Yasin, Ehtisham Ul Haq, Moazzam Ali iD, Jungsuk Kim iD, Asif Mehmood iD

Sensors · 17.07.2024 · 24 (14) · 4646

DOI: 10.3390/s24144646 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 17.07.2024. Verfasst von Sajid Ur Rehman und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Creative Technologies, Air University, Islamabad 44000, Pakistan. Hauptthemen laut MEDLINE: Algorithms, Human Activities. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea, Korea Institute of Industrial Technology Evaluation and Management (KEIT).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human activity recognition (HAR) is pivotal in advancing applications ranging from healthcare monitoring to interactive gaming. Traditional HAR systems, primarily relying on single data sources, face limitations in capturing the full spectrum of human activities. This study introduces a comprehensive approach to HAR by integrating two critical modalities: RGB imaging and advanced pose estimation features. Our methodology leverages the strengths of each modality to overcome the drawbacks of unimodal systems, providing a richer and more accurate representation of activities. We propose a two-stream network that processes skeletal and RGB data in parallel, enhanced by pose estimation techniques for refined feature extraction. The integration of these modalities is facilitated through advanced fusion algorithms, significantly improving recognition accuracy. Extensive experiments conducted on the UTD multimodal human action dataset (UTD MHAD) demonstrate that the proposed approach exceeds the performance of existing state-of-the-art algorithms, yielding improved outcomes. This study not only sets a new benchmark for HAR systems but also highlights the importance of feature engineering in capturing the complexity of human movements and the integration of optimal features. Our findings pave the way for more sophisticated, reliable, and applicable HAR systems in real-world scenarios.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AlgorithmsHuman ActivitiesHumansImage Processing, Computer-AssistedMovementPosturePattern Recognition, Automated

Stichwörter der Autor:innen

2 + 1 dimensional convolutional neural network (2 + 1D CNN)Human Activity Recognition (HAR)UTD Multimodal Human Action Dataset (UTD MHAD)deep learningmultimodal fusionpose estimationskeletal extractionspatiotemporal feature extractiontwo-stream network

Autor:innen und Institutionen

  • Department of Creative Technologies, Air University, Islamabad 44000, Pakistan.
  • Department of Biomedical Engineering, College of IT Convergence, Gachon University, 1342 Seongnamdaero, Sujeong-gu, Seongnam-si 13120, Republic of Korea.
  • Research and Development Laboratory, Cellico Company, Seongnam-si 13449, Republic of Korea.

ORCID-Profile: Sajid Ur Rehman, Moazzam Ali, Jungsuk Kim, Asif Mehmood

Förderung

  • National Research Foundation of Korea
  • Korea Institute of Industrial Technology Evaluation and Management (KEIT)

Zitieren

APA

Rehman, S. U., Yasin, A. U., Ul Haq, E., Ali, M., Kim, J., & Mehmood, A. (2024). Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation. Sensors, 24(14), 4646. https://doi.org/10.3390/s24144646

Vancouver

Rehman SU, Yasin AU, Ul Haq E, Ali M, Kim J, Mehmood A. Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation. Sensors. 2024;24(14):4646. doi:10.3390/s24144646

BibTeX
@article{Rehman2024,
  author = {Rehman, Sajid Ur and Yasin, Aman Ullah and Ul Haq, Ehtisham and Ali, Moazzam and Kim, Jungsuk and Mehmood, Asif},
  title = {Enhancing Human Activity Recognition through Integrated Multimodal Analysis: A Focus on RGB Imaging, Skeletal Tracking, and Pose Estimation},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {14},
  pages = {4646},
  doi = {10.3390/s24144646}
}
Kostenlos starten →