Evaluating Facial Expression Recognition Datasets for Deep Learning: A Benchmark Study With Novel Similarity Metrics
IEEE Transactions on Affective Computing · 2026-07 · 17 (3) · 3717-3729
DOI: 10.1109/taffc.2026.3693316 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2026-07. Verfasst von F. Xavier Gaya-Morey und 3 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This study investigates the key characteristics and suitability of widely used Facial Expression Recognition (FER) datasets for training deep learning models. In the field of affective computing, FER is essential for interpreting human emotions, yet the performance of FER systems is highly contingent on the quality and diversity of the underlying datasets. To address this issue, we compiled and analyzed 24 FER datasets, including those targeting specific age groups such as children, adults, and the elderly, and processed them through a comprehensive normalization pipeline. In addition, we enriched the datasets with automatic annotations for age and gender, enabling a more nuanced evaluation of their demographic properties. To further assess dataset efficacy, we introduce three novel metricsLocal, Global, and Paired Similarity, which quantitatively measure dataset difficulty, generalization capability, and cross-dataset transferability. Benchmark experiments using state-of-the-art neural networks reveal that large-scale, automatically collected datasets (e.g., AffectNet, FER2013) tend to generalize better, despite issues with labeling noise and demographic biases, whereas controlled datasets offer higher annotation quality but limited variability. Our findings provide actionable recommendations for dataset selection and design, advancing the development of more robust, fair, and effective FER systems.
Zitieren
APA
Gaya-Morey, F. X., Manresa-Yee, C., Martinie, C., & Buades-Rubio, J. M. (2026). Evaluating Facial Expression Recognition Datasets for Deep Learning: A Benchmark Study With Novel Similarity Metrics. IEEE Transactions on Affective Computing, 17(3), 3717-3729. https://doi.org/10.1109/taffc.2026.3693316
Vancouver
Gaya-Morey FX, Manresa-Yee C, Martinie C, Buades-Rubio JM. Evaluating Facial Expression Recognition Datasets for Deep Learning: A Benchmark Study With Novel Similarity Metrics. IEEE Transactions on Affective Computing. 2026;17(3):3717-3729. doi:10.1109/taffc.2026.3693316
BibTeX
@article{GayaMorey2026,
author = {Gaya-Morey, F. Xavier and Manresa-Yee, Cristina and Martinie, Celia and Buades-Rubio, Jose M.},
title = {Evaluating Facial Expression Recognition Datasets for Deep Learning: A Benchmark Study With Novel Similarity Metrics},
journal = {IEEE Transactions on Affective Computing},
year = {2026},
volume = {17},
number = {3},
pages = {3717--3729},
doi = {10.1109/taffc.2026.3693316}
}
EUVIMED