EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Nianfeng Li iD, Yongyuan Huang, Zhenyan Wang, Ziyao Fan, Xinyuan Li, Zhiguo Xiao

Sensors · 26.06.2024 · 24 (13) · 4153

DOI: 10.3390/s24134153 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 26.06.2024. Verfasst von Nianfeng Li und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Computer Science and Technology, Changchun University, No. 6543, Satellite Road, Changchun 130022, China. Hauptthemen laut MEDLINE: Facial Expression, Neural Networks, Computer. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Jilin Provincial Science and Technology Development Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Convolutional neural networks (CNNs) have made significant progress in the field of facial expression recognition (FER). However, due to challenges such as occlusion, lighting variations, and changes in head pose, facial expression recognition in real-world environments remains highly challenging. At the same time, methods solely based on CNN heavily rely on local spatial features, lack global information, and struggle to balance the relationship between computational complexity and recognition accuracy. Consequently, the CNN-based models still fall short in their ability to address FER adequately. To address these issues, we propose a lightweight facial expression recognition method based on a hybrid vision transformer. This method captures multi-scale facial features through an improved attention module, achieving richer feature integration, enhancing the network’s perception of key facial expression regions, and improving feature extraction capabilities. Additionally, to further enhance the model’s performance, we have designed the patch dropping (PD) module. This module aims to emulate the attention allocation mechanism of the human visual system for local features, guiding the network to focus on the most discriminative features, reducing the influence of irrelevant features, and intuitively lowering computational costs. Extensive experiments demonstrate that our approach significantly outperforms other methods, achieving an accuracy of 86.51% on RAF-DB and nearly 70% on FER2013, with a model size of only 3.64 MB. These results demonstrate that our method provides a new perspective for the field of facial expression recognition.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Facial ExpressionNeural Networks, ComputerHumansAutomated Facial RecognitionAlgorithmsImage Processing, Computer-AssistedFacePattern Recognition, Automated

Stichwörter der Autor:innen

attention modulefacial expression recognitionlightweight networktransformer

Autor:innen und Institutionen

  • College of Computer Science and Technology, Changchun University, No. 6543, Satellite Road, Changchun 130022, China.
  • School of Computer Science Technology, Beijing Institute of Technology, Beijing 100811, China.

ORCID-Profile: Nianfeng Li

Förderung

  • Jilin Provincial Science and Technology Development Program

Zitieren

APA

Li, N., Huang, Y., Wang, Z., Fan, Z., Li, X., & Xiao, Z. (2024). Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition. Sensors, 24(13), 4153. https://doi.org/10.3390/s24134153

Vancouver

Li N, Huang Y, Wang Z, Fan Z, Li X, Xiao Z. Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition. Sensors. 2024;24(13):4153. doi:10.3390/s24134153

BibTeX
@article{Li2024,
  author = {Li, Nianfeng and Huang, Yongyuan and Wang, Zhenyan and Fan, Ziyao and Li, Xinyuan and Xiao, Zhiguo},
  title = {Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {13},
  pages = {4153},
  doi = {10.3390/s24134153}
}
Kostenlos starten →