Predicting depression by using a novel deep learning model and video-audio-text multimodal data
Frontiers in Psychiatry · 24.09.2025 · 16
DOI: 10.3389/fpsyt.2025.1602650 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Psychiatry, online erschienen am 24.09.2025. Verfasst von Yifu Li und 6 weiteren Autor:innen. Beteiligte Einrichtung: College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Themen: Depression
Abstract (Originalsprache)
Objective
Depression is a prevalent mental health disorder affecting millions of people. Traditional diagnostic methods primarily rely on self-reported questionnaires and clinical interviews, which can be subjective and vary significantly between individuals. This paper introduces the Integrative Multimodal Depression Detection Network (IMDD-Net), a novel deep-learning framework designed to enhance the accuracy of depression evaluation by leveraging both local and global features from video, audio, and text cues.
Methods
The IMDD-Net integrates these multimodal data streams using the Kronecker product for multimodal fusion, facilitating deep interactions between modalities. Within the audio modality, Mel Frequency Cepstrum Coefficient (MFCC) and extended Geneva Minimalistic Acoustic Parameter Set (eGeMAPS) features capture local and global acoustic properties, respectively. For video data, the TimeSformer network extracts both fine-grained and broad temporal features, while the text modality utilizes a pre-trained BERT model to obtain comprehensive contextual information. The IMDD-Net’s architecture effectively combines these diverse data types to provide a holistic analysis of depressive symptoms.
Results
Experimental results on the AVEC 2014 dataset demonstrate that the IMDD-Net achieves state-of-the-art performance in predicting Beck Depression Inventory-II (BDI-II) scores, with a Root Mean Square Error (RMSE) of 7.55 and a Mean Absolute Error (MAE) of 5.75. A classification to identify potential depression subjects can achieve an accuracy of 0.79.
Conclusion
These results underscore the robustness and precision of the IMDD-Net, highlighting the importance of integrating local and global features across multiple modalities for accurate depression prediction.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China.
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, China.
- Department of Psychology, The People's Hospital of Liaoning Province, Shenyang, China.
- Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ, United States.
Zitieren
APA
Li, Y., Yang, X., Zhao, M., Wang, J., Yao, Y., Qian, W., & Qi, S. (2025). Predicting depression by using a novel deep learning model and video-audio-text multimodal data. Frontiers in Psychiatry, 16. https://doi.org/10.3389/fpsyt.2025.1602650
Vancouver
Li Y, Yang X, Zhao M, Wang J, Yao Y, Qian W, et al. Predicting depression by using a novel deep learning model and video-audio-text multimodal data. Frontiers in Psychiatry. 2025;16. doi:10.3389/fpsyt.2025.1602650
BibTeX
@article{Li2025,
author = {Li, Yifu and Yang, Xueping and Zhao, Meng and Wang, Jiangtao and Yao, Yudong and Qian, Wei and Qi, Shouliang},
title = {Predicting depression by using a novel deep learning model and video-audio-text multimodal data},
journal = {Frontiers in Psychiatry},
year = {2025},
volume = {16},
doi = {10.3389/fpsyt.2025.1602650}
}
EUVIMED