EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Enhancing Facial Expression Recognition through Light Field Cameras

Sabrine Djedjiga Oucherif iD, Mohamad Motasem Nawaf iD, Jean-Marc Boï, Lionel Nicod iD, Elodie Mallor, Séverine Dubuisson iD, Djamal Merad iD

Sensors · 2024

DOI: 10.3390/s24175724 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Sabrine Djedjiga Oucherif und 6 weiteren Autor:innen. Beteiligte Einrichtung: Institut de Mathématiques de Marseille (IMM), CNRS, Aix-Marseille University, 13009 Marseille, France. Hauptthemen laut MEDLINE: Facial Expression, Neural Networks, Computer. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper, we study facial expression recognition (FER) using three modalities obtained from a light field camera: sub-aperture (SA), depth map, and all-in-focus (AiF) images. Our objective is to construct a more comprehensive and effective FER system by investigating multimodal fusion strategies. For this purpose, we employ EfficientNetV2-S, pre-trained on AffectNet, as our primary convolutional neural network. This model, combined with a BiGRU, is used to process SA images. We evaluate various fusion techniques at both decision and feature levels to assess their effectiveness in enhancing FER accuracy. Our findings show that the model using SA images surpasses state-of-the-art performance, achieving 88.13% ± 7.42% accuracy under the subject-specific evaluation protocol and 91.88% ± 3.25% under the subject-independent evaluation protocol. These results highlight our model’s potential in enhancing FER accuracy and robustness, outperforming existing methods. Furthermore, our multimodal fusion approach, integrating SA, AiF, and depth images, demonstrates substantial improvements over unimodal models. The decision-level fusion strategy, particularly using average weights, proved most effective, achieving 90.13% ± 4.95% accuracy under the subject-specific evaluation protocol and 93.33% ± 4.92% under the subject-independent evaluation protocol. This approach leverages the complementary strengths of each modality, resulting in a more comprehensive and accurate FER system.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Facial ExpressionNeural Networks, ComputerHumansImage Processing, Computer-AssistedAutomated Facial RecognitionAlgorithmsPattern Recognition, Automated

Stichwörter der Autor:innen

facial expression recognitionlight field camerasmultimodality

Autor:innen und Institutionen

  • Institut de Mathématiques de Marseille (IMM), CNRS, Aix-Marseille University, 13009 Marseille, France.
  • Laboratoire d'Informatique et des Systèmes (LIS), CNRS, Aix-Marseille University, 13009 Marseille, France.
  • Centre d'Etudes et de Recherche en Gestion d'Aix-Marseille (CERGAM), Aix-Marseille University, 13013 Marseille, France.

ORCID-Profile: Sabrine Djedjiga Oucherif, Mohamad Motasem Nawaf, Lionel Nicod, Séverine Dubuisson, Djamal Merad

Zitieren

APA

Oucherif, S. D., Nawaf, M. M., Boï, J. M., Nicod, L., Mallor, E., Dubuisson, S., & Merad, D. (2024). Enhancing Facial Expression Recognition through Light Field Cameras. Sensors. https://doi.org/10.3390/s24175724

Vancouver

Oucherif SD, Nawaf MM, Boï JM, Nicod L, Mallor E, Dubuisson S, et al. Enhancing Facial Expression Recognition through Light Field Cameras. Sensors. 2024. doi:10.3390/s24175724

BibTeX
@article{Oucherif2024,
  author = {Oucherif, Sabrine Djedjiga and Nawaf, Mohamad Motasem and Boï, Jean-Marc and Nicod, Lionel and Mallor, Elodie and Dubuisson, Séverine and Merad, Djamal},
  title = {Enhancing Facial Expression Recognition through Light Field Cameras},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24175724}
}
Kostenlos starten →