EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Two-Stream Method for Human Action Recognition Using Facial Action Cues

Zhimao Lai iD, Yan Zhang, Xiubo Liang

Sensors · 2024

DOI: 10.3390/s24216817 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Zhimao Lai und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Immigration Administration (Guangzhou), China People's Police University, Guangzhou 510663, China. Hauptthemen laut MEDLINE: Cues, Face. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Fund Cultivation Project from China People's Police University, National Fund Cultivation Project from China People’s Police University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human action recognition (HAR) is a critical area in computer vision with wide-ranging applications, including video surveillance, healthcare monitoring, and abnormal behavior detection. Current HAR methods predominantly rely on full-body data, which can limit their effectiveness in real-world scenarios where occlusion is common. In such situations, the face often remains visible, providing valuable cues for action recognition. This paper introduces Face in Action (FIA), a novel two-stream method that leverages facial action cues for robust action recognition under conditions of significant occlusion. FIA consists of an RGB stream and a landmark stream. The RGB stream processes facial image sequences using a fine-spatio-multitemporal (FSM) 3D convolution module, which employs smaller spatial receptive fields to capture detailed local facial movements and larger temporal receptive fields to model broader temporal dynamics. The landmark stream processes facial landmark sequences using a normalized temporal attention (NTA) module within an NTA-GCN block, enhancing the detection of key facial frames and improving overall recognition accuracy. We validate the effectiveness of FIA using the NTU RGB+D and NTU RGB+D 120 datasets, focusing on action categories related to medical conditions. Our experiments demonstrate that FIA significantly outperforms existing methods in scenarios with extensive occlusion, highlighting its potential for practical applications in surveillance and healthcare settings.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

CuesFaceHumansPattern Recognition, AutomatedAlgorithmsImage Processing, Computer-AssistedHuman Activities

Stichwörter der Autor:innen

deep learningfacial actionfine-spatio-multitemporalhuman action recognitionnormalized temporal attention

Autor:innen und Institutionen

  • School of Immigration Administration (Guangzhou), China People's Police University, Guangzhou 510663, China.
  • School of Immigration Administration, China People's Police University, Langfang 065000, China.

ORCID-Profile: Zhimao Lai

Förderung

  • National Key Research and Development Program of China
  • National Fund Cultivation Project from China People's Police University
  • National Fund Cultivation Project from China People’s Police University

Zitieren

APA

Lai, Z., Zhang, Y., & Liang, X. (2024). A Two-Stream Method for Human Action Recognition Using Facial Action Cues. Sensors. https://doi.org/10.3390/s24216817

Vancouver

Lai Z, Zhang Y, Liang X. A Two-Stream Method for Human Action Recognition Using Facial Action Cues. Sensors. 2024. doi:10.3390/s24216817

BibTeX
@article{Lai2024,
  author = {Lai, Zhimao and Zhang, Yan and Liang, Xiubo},
  title = {A Two-Stream Method for Human Action Recognition Using Facial Action Cues},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24216817}
}
Kostenlos starten →