EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods

A.M. Mutawa, Aya Hassouneh

Biomedical Signal Processing and Control · 2024-05 · 91 · 105942

DOI: 10.1016/j.bspc.2023.105942 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biomedical Signal Processing and Control, 2024-05. Verfasst von A.M. Mutawa und 1 weiteren Autor:innen. Die Arbeit wurde bisher 63-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Kuwait University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Human Machine Interface (HMI) depends on emotion detection, especially for hospitalized patients. The emergence of the fourth industrial revolution (4IR) has heightened interest in emotional intelligence in human-computer interaction (HCI). This work employs electroencephalography (EEG), an optical flow algorithm, and machine learning to create a multimodal intelligent real-time emotion recognition system. The objective is to assist hospitalized patients, disabled (deaf, dumb, and bedridden) individuals, and autistic youngsters in expressing their authentic feelings. We feed our multimodal feature fusion vector to a classifier with long short-term memory (LSTM). We distinguish six fundamental emotions: anger, disgust, fear, sadness, joy, and surprise. The fusion feature vector is created utilizing the patient's geometrical face characteristics and EEG inputs. Utilizing fourteen EEG inputs, we use four-band relative power channels, namely alpha (8-13 Hz), beta (13-30 Hz), gamma (30-49 Hz), and theta (4-8 Hz). We achieved a maximum recognition rate of 90.25 percent using just facial landmarks and 87.25 percent using only EEG data. When both face and EEG streams are examined, we achieve 99.3 percent accuracy in a multimodal method

Förderung

  • Kuwait University

Zitieren

APA

Mutawa, A., & Hassouneh, A. (2024). Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods. Biomedical Signal Processing and Control, 91, 105942. https://doi.org/10.1016/j.bspc.2023.105942

Vancouver

Mutawa A, Hassouneh A. Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods. Biomedical Signal Processing and Control. 2024;91:105942. doi:10.1016/j.bspc.2023.105942

BibTeX
@article{Mutawa2024,
  author = {Mutawa, A.M. and Hassouneh, Aya},
  title = {Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods},
  journal = {Biomedical Signal Processing and Control},
  year = {2024},
  volume = {91},
  pages = {105942},
  doi = {10.1016/j.bspc.2023.105942}
}
Kostenlos starten →