Enhancing Emotion Recognition through Federated Learning: A Multimodal Approach with Convolutional Neural Networks
Applied Sciences · 06.02.2024 · 14 (4) · 1325
DOI: 10.3390/app14041325 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 06.02.2024. Verfasst von Nikola Simić und 6 weiteren Autor:innen. Die Arbeit wurde bisher 37-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union’s Horizon 2020 research.
Abstract (Originalsprache)
Human–machine interaction covers a range of applications in which machines should understand humans’ commands and predict their behavior. Humans commonly change their mood over time, which affects the way we interact, particularly by changing speech style and facial expressions. As interaction requires quick decisions, low latency is critical for real-time processing. Edge devices, strategically placed near the data source, minimize processing time, enabling real-time decision-making. Edge computing allows us to process data locally, thus reducing the need to send sensitive information further through the network. Despite the wide adoption of audio-only, video-only, and multimodal emotion recognition systems, there is a research gap in terms of analyzing lightweight models and solving privacy challenges to improve model performance. This motivated us to develop a privacy-preserving, lightweight, CNN-based (CNNs are frequently used for processing audio and video modalities) audiovisual emotion recognition model, deployable on constrained edge devices. The model is further paired with a federated learning protocol to preserve the privacy of local clients on edge devices and improve detection accuracy. The results show that the adoption of federated learning improved classification accuracy by ~2%, as well as that the proposed federated learning-based model provides competitive performance compared to other baseline audiovisual emotion recognition models.
Förderung
- European Union’s Horizon 2020 research
Zitieren
APA
Simić, N., Suzić, S., Milošević, N., Stanojev, V., Nosek, T., Popović, B., & Bajović, D. (2024). Enhancing Emotion Recognition through Federated Learning: A Multimodal Approach with Convolutional Neural Networks. Applied Sciences, 14(4), 1325. https://doi.org/10.3390/app14041325
Vancouver
Simić N, Suzić S, Milošević N, Stanojev V, Nosek T, Popović B, et al. Enhancing Emotion Recognition through Federated Learning: A Multimodal Approach with Convolutional Neural Networks. Applied Sciences. 2024;14(4):1325. doi:10.3390/app14041325
BibTeX
@article{Simi2024,
author = {Simić, Nikola and Suzić, Siniša and Milošević, Nemanja and Stanojev, Vuk and Nosek, Tijana and Popović, Branislav and Bajović, Dragana},
title = {Enhancing Emotion Recognition through Federated Learning: A Multimodal Approach with Convolutional Neural Networks},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {4},
pages = {1325},
doi = {10.3390/app14041325}
}
EUVIMED