Depression Scale Dictionary Decomposition Framework for Multimodal Automatic Depression Level Prediction
IEEE Transactions on Circuits and Systems for Video Technology · 2025-06 · 35 (6) · 6195-6210
DOI: 10.1109/tcsvt.2025.3533480 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2025-06. Verfasst von Mingyue Niu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 72 Literaturangaben. Gefördert von: Basic Innovation and Scientific Research Cultivation Project of Yanshan University, S&T Program of Hebei, National Natural Science Foundation of China.
Themen: Depression
Abstract (Originalsprache)
Currently, many researchers aim to achieve automatic depression level prediction via speech and video behavior analysis. However, previous works have struggled to decompose audio and video sequences into the information related to and unrelated to depression scores, hindering the model’s perception of depression cues. Besides, previous works implement multimodal fusion using attention mechanisms or linear layers, but failed to simultaneously consider the Euclidean relationship among tokens and the non-Euclidean relationship among channels, which bring limitations in capturing depression cues. In response to the above issues, we propose a depression scale dictionary decomposition framework, which mainly includes a Bidirectional Dictionary Decomposition (BDD) module and a Bidirectional Multimodal Fusion (BMF) module. The BDD module can use the dictionaries generated based on the depression scale to semantically decompose audio and video sequences into the information related to and unrelated to depression scores along token and channel dimensions for promoting depression cue perception. Moreover, considering the respective characteristics of tokens and channels, the BMF module uses linear layers and graph convolution to achieve cross-modal mixing, which is used to aggregate audio and video sequences for predicting depression levels. The validation on AVEC 2013, AVEC 2014 and DAIC-WOZ datasets demonstrates our method’s superiority.
Förderung
- Basic Innovation and Scientific Research Cultivation Project of Yanshan University
- S&T Program of Hebei
- National Natural Science Foundation of China
- Central Government Guidance Local Science and Technology Development Fund Project
Zitieren
APA
Niu, M., Wang, X., Gong, J., Liu, B., Tao, J., & Schuller, B. W. (2025). Depression Scale Dictionary Decomposition Framework for Multimodal Automatic Depression Level Prediction. IEEE Transactions on Circuits and Systems for Video Technology, 35(6), 6195-6210. https://doi.org/10.1109/tcsvt.2025.3533480
Vancouver
Niu M, Wang X, Gong J, Liu B, Tao J, Schuller BW. Depression Scale Dictionary Decomposition Framework for Multimodal Automatic Depression Level Prediction. IEEE Transactions on Circuits and Systems for Video Technology. 2025;35(6):6195-6210. doi:10.1109/tcsvt.2025.3533480
BibTeX
@article{Niu2025,
author = {Niu, Mingyue and Wang, Xu and Gong, Jibing and Liu, Bin and Tao, Jianhua and Schuller, Björn W.},
title = {Depression Scale Dictionary Decomposition Framework for Multimodal Automatic Depression Level Prediction},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2025},
volume = {35},
number = {6},
pages = {6195--6210},
doi = {10.1109/tcsvt.2025.3533480}
}
EUVIMED