Effect of observer’s cultural background and masking condition of target face on facial expression recognition for machine-learning dataset
PLOS ONE · 2024
DOI: 10.1371/journal.pone.0313029 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Masahito Nagata und 1 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Environment and Information Sciences, Yokohama National University, Yokohama, Kanagawa, Japan. Hauptthemen laut MEDLINE: Emotions, Facial Expression, Facial Recognition, Machine Learning. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JST Center of Innovation (COI) STREAM.
Abstract (Originalsprache)
Facial expression recognition (FER) is significantly influenced by the cultural background (CB) of observers and the masking conditions of the target face. This study aimed to clarify these factors’ impact on FER, particularly in machine-learning datasets, increasingly used in human-computer interaction and automated systems. We conducted an FER experiment with East Asian participants and compared the results with the FERPlus dataset, evaluated by Western raters. Our novel analysis approach focused on variability between images and participants within a "majority" category and the eye-opening rate of target faces, providing a deeper understanding of FER processes. Notable findings were differences in "fear" perception between East Asians and Westerners, with East Asians more likely to interpret "fear" as "surprise." Masking conditions significantly affected emotion categorization, with "fear" perceived by East Asians for non-masked faces interpreted as "surprise" for masked faces. Then, the emotion labels were perceived as different emotions across categories in the masking condition, rather than simply lower recognition rates or confusion as in existing studies. Additionally, "sadness" perceived by Westerners was often interpreted as "disgust" by East Asians. These results suggest that one-to-one network learning models, commonly trained using majority labels, might overlook important minority response information, potentially leading to biases in automated FER systems. In conclusion, FER dataset characteristics differ depending on the target face’s masking condition and the diversity among evaluation groups. This study highlights the need to consider these factors in machine-learning-based FER that relies on human-judged labels, to contribute to the development of more nuanced and fair automated FER systems. Our findings emphasize the novelty of our approach compared to existing studies and the importance of incorporating a broader range of human variability in FER research, setting the stage for future evaluations of machine learning classifiers on similar data.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Faculty of Environment and Information Sciences, Yokohama National University, Yokohama, Kanagawa, Japan.
ORCID-Profile: Masahito Nagata
Förderung
- JST Center of Innovation (COI) STREAM
Zitieren
APA
Nagata, M., & Okajima, K. (2024). Effect of observer’s cultural background and masking condition of target face on facial expression recognition for machine-learning dataset. PLOS ONE. https://doi.org/10.1371/journal.pone.0313029
Vancouver
Nagata M, Okajima K. Effect of observer’s cultural background and masking condition of target face on facial expression recognition for machine-learning dataset. PLOS ONE. 2024. doi:10.1371/journal.pone.0313029
BibTeX
@article{Nagata2024,
author = {Nagata, Masahito and Okajima, Katsunori},
title = {Effect of observer’s cultural background and masking condition of target face on facial expression recognition for machine-learning dataset},
journal = {PLOS ONE},
year = {2024},
doi = {10.1371/journal.pone.0313029}
}
EUVIMED