Late Fusion Model for Emotion Recognition from Facial Expressions and Biosignals in a Dataset of Children with Autism Spectrum Disorder
Sensors · 2025
DOI: 10.3390/s25247485 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Dominika Kiejdo und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Electronics, Telecommunications and Informatics, Gdańsk University of Technology, 80-233 Gdańsk, Poland. Hauptthemen laut MEDLINE: Autism Spectrum Disorder, Facial Expression, Emotions. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Themen: Kinderkrankenpflege
Abstract (Originalsprache)
Children with autism spectrum disorder (ASD) often display atypical emotional expressions and physiological responses, making emotion recognition challenging. This study proposes a multimodal recognition model employing a late fusion framework combining facial expression with physiological measures: electrodermal activity (EDA), temperature (TEMP), and heart rate (HR). Emotional states are annotated using two complementary schemes derived from a shared set of labels. Three annotators provide one categorical Ekman emotion for each timestamp. From these annotations, a majority-vote label identifies the dominant emotion, while a proportional distribution reflects the likelihood of each emotion based on the relative frequency of the annotators’ selections. Separate machine learning models are trained for each modality and for each annotation scheme, and their outputs are integrated through decision-level fusion. A distinct decision-level fusion model is constructed for each annotation scheme, ensuring that both the categorical and likelihood-based representations are optimally combined. The experiments on the EMBOA dataset, collected within the project “Affective loop in Socially Assistive Robotics as an intervention tool for children with autism”, show that the late fusion model achieves higher accuracy and robustness than unimodal baselines. The system attains an accuracy of 68% for categorical emotion classification and 78% under the likelihood-estimation scheme. The results obtained, although lower than those reported in other studies, suggest that further research into emotion recognition in autistic children using other fusions is warranted, even in the case of datasets with a significant number of missing values and low sample representation for certain emotions.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Electronics, Telecommunications and Informatics, Gdańsk University of Technology, 80-233 Gdańsk, Poland.
ORCID-Profile: Teresa Zawadzka
Zitieren
APA
Kiejdo, D., Depka Prądzinska, M., & Zawadzka, T. (2025). Late Fusion Model for Emotion Recognition from Facial Expressions and Biosignals in a Dataset of Children with Autism Spectrum Disorder. Sensors. https://doi.org/10.3390/s25247485
Vancouver
Kiejdo D, Depka Prądzinska M, Zawadzka T. Late Fusion Model for Emotion Recognition from Facial Expressions and Biosignals in a Dataset of Children with Autism Spectrum Disorder. Sensors. 2025. doi:10.3390/s25247485
BibTeX
@article{Kiejdo2025,
author = {Kiejdo, Dominika and Depka Prądzinska, Monika and Zawadzka, Teresa},
title = {Late Fusion Model for Emotion Recognition from Facial Expressions and Biosignals in a Dataset of Children with Autism Spectrum Disorder},
journal = {Sensors},
year = {2025},
doi = {10.3390/s25247485}
}
EUVIMED