EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Facial expression recognition through muscle synergies and estimation of facial keypoint displacements through a skin-musculoskeletal model using facial sEMG signals

Lun Shu, Victor R Barradas, Zixuan Qin, Yasuharu Koike

Frontiers in Bioengineering and Biotechnology · 12.02.2025 · 13

DOI: 10.3389/fbioe.2025.1490919 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Bioengineering and Biotechnology, online erschienen am 12.02.2025. Verfasst von Lun Shu und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Information and Communications Engineering, Institute of Science Tokyo, Yokohama, Japan. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 98 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The development of facial expression recognition (FER) and facial expression generation (FEG) systems is essential to enhance human-robot interactions (HRI). The facial action coding system is widely used in FER and FEG tasks, as it offers a framework to relate the action of facial muscles and the resulting facial motions to the execution of facial expressions. However, most FER and FEG studies are based on measuring and analyzing facial motions, leaving the facial muscle component relatively unexplored. This study introduces a novel framework using surface electromyography (sEMG) signals from facial muscles to recognize facial expressions and estimate the displacement of facial keypoints during the execution of the expressions. For the facial expression recognition task, we studied the coordination patterns of seven muscles, expressed as three muscle synergies extracted through non-negative matrix factorization, during the execution of six basic facial expressions. Muscle synergies are groups of muscles that show coordinated patterns of activity, as measured by their sEMG signals, and are hypothesized to form the building blocks of human motor control. We then trained two classifiers for the facial expressions based on extracted features from the sEMG signals and the synergy activation coefficients of the extracted muscle synergies, respectively. The accuracy of both classifiers outperformed other systems that use sEMG to classify facial expressions, although the synergy-based classifier performed marginally worse than the sEMG-based one (classification accuracy: synergy-based 97.4%, sEMG-based 99.2%). However, the extracted muscle synergies revealed common coordination patterns between different facial expressions, allowing a low-dimensional quantitative visualization of the muscle control strategies involved in human facial expression generation. We also developed a skin-musculoskeletal model enhanced by linear regression (SMSM-LRM) to estimate the displacement of facial keypoints during the execution of a facial expression based on sEMG signals. Our proposed approach achieved a relatively high fidelity in estimating these displacements (NRMSE 0.067). We propose that the identified muscle synergies could be used in combination with the SMSM-LRM model to generate motor commands and trajectories for desired facial displacements, potentially enabling the generation of more natural facial expressions in social robotics and virtual reality.

Schlagwörter

Stichwörter der Autor:innen

facial expression recognitionfacial keypoints estimationmuscle synergymusculoskeletal modelsEMG

Autor:innen und Institutionen

  • Department of Information and Communications Engineering, Institute of Science Tokyo, Yokohama, Japan.
  • Institute of Integrated Research, Institute of Science Tokyo, Yokohama, Japan.

Zitieren

APA

Shu, L., Barradas, V. R., Qin, Z., & Koike, Y. (2025). Facial expression recognition through muscle synergies and estimation of facial keypoint displacements through a skin-musculoskeletal model using facial sEMG signals. Frontiers in Bioengineering and Biotechnology, 13. https://doi.org/10.3389/fbioe.2025.1490919

Vancouver

Shu L, Barradas VR, Qin Z, Koike Y. Facial expression recognition through muscle synergies and estimation of facial keypoint displacements through a skin-musculoskeletal model using facial sEMG signals. Frontiers in Bioengineering and Biotechnology. 2025;13. doi:10.3389/fbioe.2025.1490919

BibTeX
@article{Shu2025,
  author = {Shu, Lun and Barradas, Victor R and Qin, Zixuan and Koike, Yasuharu},
  title = {Facial expression recognition through muscle synergies and estimation of facial keypoint displacements through a skin-musculoskeletal model using facial sEMG signals},
  journal = {Frontiers in Bioengineering and Biotechnology},
  year = {2025},
  volume = {13},
  doi = {10.3389/fbioe.2025.1490919}
}
Kostenlos starten →