EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition

Yerim Ji, Suh-Yeon Dong

IEEE Access · 2024 · 12 · 92175-92180

DOI: 10.1109/access.2024.3422403 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2024. Verfasst von Yerim Ji und 1 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Research Foundation of Korea, Institute of Information & Communications Technology Planning & Evaluation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Building a robust facial expression recognition (FER) system remains a challenging problem due to the emotional ambiguity of facial expressions. Recent approaches employ both facial expressions and physiological signals to design multi-modal emotion recognition systems. However, these approaches require physical contact with the skin as they need to use sensor modalities. To meet the demands for a non-contact emotion recognition system, we use a convolutional recurrent neural network (CRNN) to extract facial features and utilize these features for estimating the heart rate (HR) from face image sequences. In particular, unlike the conventional feature fusion method, we propose a multi-task learning (MTL) framework to simultaneously predict the emotion and HR from face image sequences using a single model. Experiments on the DEAP and MAHNOB-HCI datasets demonstrate that the proposed multi-task framework improves FER accuracy by up to 6.85% and achieves superior performance against the state-of-the-art methods.

Förderung

  • National Research Foundation of Korea
  • Institute of Information & Communications Technology Planning & Evaluation

Zitieren

APA

Ji, Y., & Dong, S. (2024). Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition. IEEE Access, 12, 92175-92180. https://doi.org/10.1109/access.2024.3422403

Vancouver

Ji Y, Dong S. Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition. IEEE Access. 2024;12:92175-92180. doi:10.1109/access.2024.3422403

BibTeX
@article{Ji2024,
  author = {Ji, Yerim and Dong, Suh-Yeon},
  title = {Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition},
  journal = {IEEE Access},
  year = {2024},
  volume = {12},
  pages = {92175--92180},
  doi = {10.1109/access.2024.3422403}
}
Kostenlos starten →