Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition
IEEE Access · 2024 · 12 · 92175-92180
DOI: 10.1109/access.2024.3422403 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2024. Verfasst von Yerim Ji und 1 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Research Foundation of Korea, Institute of Information & Communications Technology Planning & Evaluation.
Abstract (Originalsprache)
Building a robust facial expression recognition (FER) system remains a challenging problem due to the emotional ambiguity of facial expressions. Recent approaches employ both facial expressions and physiological signals to design multi-modal emotion recognition systems. However, these approaches require physical contact with the skin as they need to use sensor modalities. To meet the demands for a non-contact emotion recognition system, we use a convolutional recurrent neural network (CRNN) to extract facial features and utilize these features for estimating the heart rate (HR) from face image sequences. In particular, unlike the conventional feature fusion method, we propose a multi-task learning (MTL) framework to simultaneously predict the emotion and HR from face image sequences using a single model. Experiments on the DEAP and MAHNOB-HCI datasets demonstrate that the proposed multi-task framework improves FER accuracy by up to 6.85% and achieves superior performance against the state-of-the-art methods.
Förderung
- National Research Foundation of Korea
- Institute of Information & Communications Technology Planning & Evaluation
Zitieren
APA
Ji, Y., & Dong, S. (2024). Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition. IEEE Access, 12, 92175-92180. https://doi.org/10.1109/access.2024.3422403
Vancouver
Ji Y, Dong S. Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition. IEEE Access. 2024;12:92175-92180. doi:10.1109/access.2024.3422403
BibTeX
@article{Ji2024,
author = {Ji, Yerim and Dong, Suh-Yeon},
title = {Multi-Task Learning by Leveraging Non-Contact Heart Rate for Robust Facial Emotion Recognition},
journal = {IEEE Access},
year = {2024},
volume = {12},
pages = {92175--92180},
doi = {10.1109/access.2024.3422403}
}
EUVIMED