A Two-Stream Method for Human Action Recognition Using Facial Action Cues
Sensors · 2024
DOI: 10.3390/s24216817 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Zhimao Lai und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Immigration Administration (Guangzhou), China People's Police University, Guangzhou 510663, China. Hauptthemen laut MEDLINE: Cues, Face. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Fund Cultivation Project from China People's Police University, National Fund Cultivation Project from China People’s Police University.
Abstract (Originalsprache)
Human action recognition (HAR) is a critical area in computer vision with wide-ranging applications, including video surveillance, healthcare monitoring, and abnormal behavior detection. Current HAR methods predominantly rely on full-body data, which can limit their effectiveness in real-world scenarios where occlusion is common. In such situations, the face often remains visible, providing valuable cues for action recognition. This paper introduces Face in Action (FIA), a novel two-stream method that leverages facial action cues for robust action recognition under conditions of significant occlusion. FIA consists of an RGB stream and a landmark stream. The RGB stream processes facial image sequences using a fine-spatio-multitemporal (FSM) 3D convolution module, which employs smaller spatial receptive fields to capture detailed local facial movements and larger temporal receptive fields to model broader temporal dynamics. The landmark stream processes facial landmark sequences using a normalized temporal attention (NTA) module within an NTA-GCN block, enhancing the detection of key facial frames and improving overall recognition accuracy. We validate the effectiveness of FIA using the NTU RGB+D and NTU RGB+D 120 datasets, focusing on action categories related to medical conditions. Our experiments demonstrate that FIA significantly outperforms existing methods in scenarios with extensive occlusion, highlighting its potential for practical applications in surveillance and healthcare settings.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Immigration Administration (Guangzhou), China People's Police University, Guangzhou 510663, China.
- School of Immigration Administration, China People's Police University, Langfang 065000, China.
ORCID-Profile: Zhimao Lai
Förderung
- National Key Research and Development Program of China
- National Fund Cultivation Project from China People's Police University
- National Fund Cultivation Project from China People’s Police University
Zitieren
APA
Lai, Z., Zhang, Y., & Liang, X. (2024). A Two-Stream Method for Human Action Recognition Using Facial Action Cues. Sensors. https://doi.org/10.3390/s24216817
Vancouver
Lai Z, Zhang Y, Liang X. A Two-Stream Method for Human Action Recognition Using Facial Action Cues. Sensors. 2024. doi:10.3390/s24216817
BibTeX
@article{Lai2024,
author = {Lai, Zhimao and Zhang, Yan and Liang, Xiubo},
title = {A Two-Stream Method for Human Action Recognition Using Facial Action Cues},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24216817}
}
EUVIMED