EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism

Cheng Peng iD, Mingqi Sun iD, Kun Zou iD, Bowen Zhang iD, Genan Dai, Ah Chung Tsoi iD

Sensors · 28.10.2024 · 24 (21) · 6912

DOI: 10.3390/s24216912 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 28.10.2024. Verfasst von Cheng Peng und 5 weiteren Autor:innen. Beteiligte Einrichtung: School of Computing, Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan 528402, China. Hauptthemen laut MEDLINE: Facial Expression. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fund for High-Level Talents Awarded by University of Electronic Science and Technology of China, Zhongshan Institute, Young Innovative Talents Project of Education Department of Guangdong Province, Featured Innovative Project of Education Department of Guangdong Province.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In studying the joint object detection and classification problem for facial expression recognition (FER) deploying the YOLOX framework, we introduce a novel feature extractor, called neighborhood coordinate attention Mamba (NCAMamba) to substitute for the original feature extractor in the Feature Pyramid Network (FPN). NCAMamba combines the background information reduction capabilities of Mamba, the local neighborhood relationship understanding of neighborhood attention, and the directional relationship understanding of coordinate attention. The resulting FER-YOLO-NCAMamba model, when applied to two unaligned FER benchmark datasets, RAF-DB and SFEW, obtains significantly improved mean average precision (mAP) scores when compared with those obtained by other state-of-the-art methods. Moreover, in ablation studies, it is found that the NCA module is relatively more important than the Visual State Space (VSS), a version of using Mamba for image processing, and in visualization studies using the grad-CAM method, it reveals that regions around the nose tip are critical to recognizing the expression; if it is too large, it may lead to erroneous prediction, while a small focused region would lead to correct recognition; this may explain why FER of unaligned faces is such a challenging problem.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Facial ExpressionHumansImage Processing, Computer-AssistedAlgorithmsAutomated Facial RecognitionPattern Recognition, AutomatedAttentionFacial Recognition

Stichwörter der Autor:innen

attentionfacial expression recognitionobject detectionvisual state space model

Autor:innen und Institutionen

  • School of Computing, Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan 528402, China.
  • School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China.
  • College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China.
  • School of Computing and Information Technology, University of Wollongong, Wollongong, NSW 2522, Australia.

ORCID-Profile: Cheng Peng, Mingqi Sun, Kun Zou, Bowen Zhang, Ah Chung Tsoi

Förderung

  • Fund for High-Level Talents Awarded by University of Electronic Science and Technology of China, Zhongshan Institute
  • Young Innovative Talents Project of Education Department of Guangdong Province
  • Featured Innovative Project of Education Department of Guangdong Province
  • special projects in key fields of universities in Guangdong Province
  • Social Welfare and Basic Research Key Project of Zhongshan City

Zitieren

APA

Peng, C., Sun, M., Zou, K., Zhang, B., Dai, G., & Tsoi, A. C. (2024). Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism. Sensors, 24(21), 6912. https://doi.org/10.3390/s24216912

Vancouver

Peng C, Sun M, Zou K, Zhang B, Dai G, Tsoi AC. Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism. Sensors. 2024;24(21):6912. doi:10.3390/s24216912

BibTeX
@article{Peng2024,
  author = {Peng, Cheng and Sun, Mingqi and Zou, Kun and Zhang, Bowen and Dai, Genan and Tsoi, Ah Chung},
  title = {Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {21},
  pages = {6912},
  doi = {10.3390/s24216912}
}
Kostenlos starten →