Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121
Brain Sciences · 15.10.2024 · 14 (10) · 1018
DOI: 10.3390/brainsci14101018 ↗Kurz gefasst
Fachartikel in der Zeitschrift Brain Sciences, online erschienen am 15.10.2024. Verfasst von Musyyab Yousufi und 2 weiteren Autor:innen. Beteiligte Einrichtung: Centre of Real Time Computer Systems, Kaunas University of Technology, 51368 Kaunas, Lithuania. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background/Objectives: This study investigates the classification of Major Depressive Disorder (MDD) using electroencephalography (EEG) Short-Time Fourier-Transform (STFT) spectrograms and audio Mel-spectrogram data of 52 subjects. The objective is to develop a multimodal classification model that integrates audio and EEG data to accurately identify depressive tendencies.
Methods
We utilized the Multimodal open dataset for Mental Disorder Analysis (MODMA) and trained a pre-trained Densenet121 model using transfer learning. Features from both the EEG and audio modalities were extracted and concatenated before being passed through the final classification layer. Additionally, an ablation study was conducted on both datasets separately.
Results
The proposed multimodal classification model demonstrated superior performance compared to existing methods, achieving an Accuracy of 97.53%, Precision of 98.20%, F1 Score of 97.76%, and Recall of 97.32%. A confusion matrix was also used to evaluate the model’s effectiveness.
Conclusions
The paper presents a robust multimodal classification approach that outperforms state-of-the-art methods with potential application in clinical diagnostics for depression assessment.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Centre of Real Time Computer Systems, Kaunas University of Technology, 51368 Kaunas, Lithuania.
ORCID-Profile: Musyyab Yousufi, Robertas Damaševičius, Rytis Maskeliūnas
Zitieren
APA
Yousufi, M., Damaševičius, R., & Maskeliūnas, R. (2024). Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121. Brain Sciences, 14(10), 1018. https://doi.org/10.3390/brainsci14101018
Vancouver
Yousufi M, Damaševičius R, Maskeliūnas R. Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121. Brain Sciences. 2024;14(10):1018. doi:10.3390/brainsci14101018
BibTeX
@article{Yousufi2024,
author = {Yousufi, Musyyab and Damaševičius, Robertas and Maskeliūnas, Rytis},
title = {Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121},
journal = {Brain Sciences},
year = {2024},
volume = {14},
number = {10},
pages = {1018},
doi = {10.3390/brainsci14101018}
}
EUVIMED