EUVIMEDObjevte EUVIMED
Open access · CC BY 4.0

Facial Landmark-Driven Keypoint Feature Extraction for Robust Facial Expression Recognition

Jaehyun So iD, Youngjoon Han iD

Sensors · 2025-06-16 · 25 (12) · 3762

DOI: 10.3390/s25123762 ↗

At a glance

Journal article in Sensors, published online on 2025-06-16. Written by Jaehyun So and 1 co-authors. Institution: Department of Electronic Engineering, Soongsil University, Seoul 06978, Republic of Korea. Major MEDLINE topics: Facial Expression, Face, Image Processing, Computer-Assisted, Automated Facial Recognition. Cited 14 times so far (Crossref); 60 references. The full text is freely available (open access, CC BY 4.0).

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Facial expression recognition (FER) is a core technology that enables computers to understand and react to human emotions. In particular, the use of face alignment algorithms as a preprocessing step in image-based FER is important for accurately normalizing face images in terms of scale, rotation, and translation to improve FER accuracy. Recently, FER studies have been actively leveraging feature maps computed by face alignment networks to enhance FER performance. However, previous studies were limited in their ability to effectively apply information from specific facial regions that are important for FER, as they either only used facial landmarks during the preprocessing step or relied solely on the feature maps from the face alignment networks. In this paper, we propose the use of Keypoint Features extracted from feature maps at the coordinates of facial landmarks. To effectively utilize Keypoint Features, we further propose a Keypoint Feature regularization method using landmark perturbation for robustness, and an attention mechanism that emphasizes all Keypoint Features using representative Keypoint Features derived from a nasal base landmark, which carries information for the whole face, to improve performance. We performed experiments on the AffectNet, RAF-DB, and FERPlus datasets using a simply designed network to validate the effectiveness of the proposed method. As a result, the proposed method achieved a performance of 68.17% on AffectNet-7, 64.87% on AffectNet-8, 93.16% on RAF-DB, and 91.44% on FERPlus. Furthermore, the network pretrained on AffectNet-8 had improved performances of 94.04% on RAF-DB and 91.66% on FERPlus. These results demonstrate that the proposed Keypoint Features can achieve comparable results to those of the existing methods, highlighting their potential for enhancing FER performance through the effective utilization of key facial region features.

Subjects

MeSH terms (MEDLINE)

Facial ExpressionFaceImage Processing, Computer-AssistedAutomated Facial RecognitionPattern Recognition, AutomatedHumansAlgorithmsEmotions

Author keywords

deep neural networkface alignmentfacial expression recognitionfeature attention

Authors and institutions

  • Department of Electronic Engineering, Soongsil University, Seoul 06978, Republic of Korea.
  • School of AI Convergence, Soongsil University, Seoul 06978, Republic of Korea.

ORCID profiles: Jaehyun So, Youngjoon Han

Cite

APA

So, J., & Han, Y. (2025). Facial Landmark-Driven Keypoint Feature Extraction for Robust Facial Expression Recognition. Sensors, 25(12), 3762. https://doi.org/10.3390/s25123762

Vancouver

So J, Han Y. Facial Landmark-Driven Keypoint Feature Extraction for Robust Facial Expression Recognition. Sensors. 2025;25(12):3762. doi:10.3390/s25123762

BibTeX
@article{So2025,
  author = {So, Jaehyun and Han, Youngjoon},
  title = {Facial Landmark-Driven Keypoint Feature Extraction for Robust Facial Expression Recognition},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {12},
  pages = {3762},
  doi = {10.3390/s25123762}
}
Start for free →