Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods
Biomedical Signal Processing and Control · 2024-05 · 91 · 105942
DOI: 10.1016/j.bspc.2023.105942 ↗Kurz gefasst
Fachartikel in der Zeitschrift Biomedical Signal Processing and Control, 2024-05. Verfasst von A.M. Mutawa und 1 weiteren Autor:innen. Die Arbeit wurde bisher 63-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Kuwait University.
Abstract (Originalsprache)
Human Machine Interface (HMI) depends on emotion detection, especially for hospitalized patients. The emergence of the fourth industrial revolution (4IR) has heightened interest in emotional intelligence in human-computer interaction (HCI). This work employs electroencephalography (EEG), an optical flow algorithm, and machine learning to create a multimodal intelligent real-time emotion recognition system. The objective is to assist hospitalized patients, disabled (deaf, dumb, and bedridden) individuals, and autistic youngsters in expressing their authentic feelings. We feed our multimodal feature fusion vector to a classifier with long short-term memory (LSTM). We distinguish six fundamental emotions: anger, disgust, fear, sadness, joy, and surprise. The fusion feature vector is created utilizing the patient's geometrical face characteristics and EEG inputs. Utilizing fourteen EEG inputs, we use four-band relative power channels, namely alpha (8-13 Hz), beta (13-30 Hz), gamma (30-49 Hz), and theta (4-8 Hz). We achieved a maximum recognition rate of 90.25 percent using just facial landmarks and 87.25 percent using only EEG data. When both face and EEG streams are examined, we achieve 99.3 percent accuracy in a multimodal method
Förderung
- Kuwait University
Zitieren
APA
Mutawa, A., & Hassouneh, A. (2024). Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods. Biomedical Signal Processing and Control, 91, 105942. https://doi.org/10.1016/j.bspc.2023.105942
Vancouver
Mutawa A, Hassouneh A. Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods. Biomedical Signal Processing and Control. 2024;91:105942. doi:10.1016/j.bspc.2023.105942
BibTeX
@article{Mutawa2024,
author = {Mutawa, A.M. and Hassouneh, Aya},
title = {Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods},
journal = {Biomedical Signal Processing and Control},
year = {2024},
volume = {91},
pages = {105942},
doi = {10.1016/j.bspc.2023.105942}
}
EUVIMED