Enhancing Facial Expression Recognition through Light Field Cameras
Sensors · 2024
DOI: 10.3390/s24175724 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Sabrine Djedjiga Oucherif und 6 weiteren Autor:innen. Beteiligte Einrichtung: Institut de Mathématiques de Marseille (IMM), CNRS, Aix-Marseille University, 13009 Marseille, France. Hauptthemen laut MEDLINE: Facial Expression, Neural Networks, Computer. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In this paper, we study facial expression recognition (FER) using three modalities obtained from a light field camera: sub-aperture (SA), depth map, and all-in-focus (AiF) images. Our objective is to construct a more comprehensive and effective FER system by investigating multimodal fusion strategies. For this purpose, we employ EfficientNetV2-S, pre-trained on AffectNet, as our primary convolutional neural network. This model, combined with a BiGRU, is used to process SA images. We evaluate various fusion techniques at both decision and feature levels to assess their effectiveness in enhancing FER accuracy. Our findings show that the model using SA images surpasses state-of-the-art performance, achieving 88.13% ± 7.42% accuracy under the subject-specific evaluation protocol and 91.88% ± 3.25% under the subject-independent evaluation protocol. These results highlight our model’s potential in enhancing FER accuracy and robustness, outperforming existing methods. Furthermore, our multimodal fusion approach, integrating SA, AiF, and depth images, demonstrates substantial improvements over unimodal models. The decision-level fusion strategy, particularly using average weights, proved most effective, achieving 90.13% ± 4.95% accuracy under the subject-specific evaluation protocol and 93.33% ± 4.92% under the subject-independent evaluation protocol. This approach leverages the complementary strengths of each modality, resulting in a more comprehensive and accurate FER system.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institut de Mathématiques de Marseille (IMM), CNRS, Aix-Marseille University, 13009 Marseille, France.
- Laboratoire d'Informatique et des Systèmes (LIS), CNRS, Aix-Marseille University, 13009 Marseille, France.
- Centre d'Etudes et de Recherche en Gestion d'Aix-Marseille (CERGAM), Aix-Marseille University, 13013 Marseille, France.
ORCID-Profile: Sabrine Djedjiga Oucherif, Mohamad Motasem Nawaf, Lionel Nicod, Séverine Dubuisson, Djamal Merad
Zitieren
APA
Oucherif, S. D., Nawaf, M. M., Boï, J. M., Nicod, L., Mallor, E., Dubuisson, S., & Merad, D. (2024). Enhancing Facial Expression Recognition through Light Field Cameras. Sensors. https://doi.org/10.3390/s24175724
Vancouver
Oucherif SD, Nawaf MM, Boï JM, Nicod L, Mallor E, Dubuisson S, et al. Enhancing Facial Expression Recognition through Light Field Cameras. Sensors. 2024. doi:10.3390/s24175724
BibTeX
@article{Oucherif2024,
author = {Oucherif, Sabrine Djedjiga and Nawaf, Mohamad Motasem and Boï, Jean-Marc and Nicod, Lionel and Mallor, Elodie and Dubuisson, Séverine and Merad, Djamal},
title = {Enhancing Facial Expression Recognition through Light Field Cameras},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24175724}
}
EUVIMED