EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN

Aniqa Arif, Yihe Wang iD, Rui Yin iD, Xiang Zhang iD, Ahmed Helmy iD

Sensors · 15.03.2024 · 24 (6) · 1889

DOI: 10.3390/s24061889 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 15.03.2024. Verfasst von Aniqa Arif und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, University of North Carolina, Charlotte, NC 28223, USA. Hauptthemen laut MEDLINE: Spectroscopy, Near-Infrared, Brain. Die Arbeit wurde bisher 62-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, University of North Carolina at Charlotte Faculty Research Grant.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Analysis of brain signals is essential to the study of mental states and various neurological conditions. The two most prevalent noninvasive signals for measuring brain activities are electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). EEG, characterized by its higher sampling frequency, captures more temporal features, while fNIRS, with a greater number of channels, provides richer spatial information. Although a few previous studies have explored the use of multimodal deep-learning models to analyze brain activity for both EEG and fNIRS, subject-independent training–testing split analysis remains underexplored. The results of the subject-independent setting directly show the model’s ability on unseen subjects, which is crucial for real-world applications. In this paper, we introduce EF-Net, a new CNN-based multimodal deep-learning model. We evaluate EF-Net on an EEG-fNIRS word generation (WG) dataset on the mental state recognition task, primarily focusing on the subject-independent setting. For completeness, we report results in the subject-dependent and subject-semidependent settings as well. We compare our model with five baseline approaches, including three traditional machine learning methods and two deep learning methods. EF-Net demonstrates superior performance in both accuracy and F1 score, surpassing these baselines. Our model achieves F1 scores of 99.36%, 98.31%, and 65.05% in the subject-dependent, subject-semidependent, and subject-independent settings, respectively, surpassing the best baseline F1 scores by 1.83%, 4.34%, and 2.13% These results highlight EF-Net’s capability to effectively learn and interpret mental states and brain activity across different and unseen subjects.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Spectroscopy, Near-InfraredBrainHumansMachine LearningElectroencephalographyHead

Stichwörter der Autor:innen

EEGbrain activity learningdeep learningfNIRSmultimodal

Autor:innen und Institutionen

  • Department of Computer Science, University of North Carolina, Charlotte, NC 28223, USA.
  • Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, FL 32611, USA.

ORCID-Profile: Yihe Wang, Rui Yin, Xiang Zhang, Ahmed Helmy

Förderung

  • National Science Foundation
  • University of North Carolina at Charlotte Faculty Research Grant

Zitieren

APA

Arif, A., Wang, Y., Yin, R., Zhang, X., & Helmy, A. (2024). EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN. Sensors, 24(6), 1889. https://doi.org/10.3390/s24061889

Vancouver

Arif A, Wang Y, Yin R, Zhang X, Helmy A. EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN. Sensors. 2024;24(6):1889. doi:10.3390/s24061889

BibTeX
@article{Arif2024,
  author = {Arif, Aniqa and Wang, Yihe and Yin, Rui and Zhang, Xiang and Helmy, Ahmed},
  title = {EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {6},
  pages = {1889},
  doi = {10.3390/s24061889}
}
Kostenlos starten →