A theory-guided multimodal framework for modeling internalizing distress using linguistic and facial-affective indicators
Multidisciplinary Science Journal · 26.10.2026 · 9 (4) · 2027349
DOI: 10.31893/multiscience.2027349 ↗Kurz gefasst
Fachartikel in der Zeitschrift Multidisciplinary Science Journal, online erschienen am 26.10.2026. Verfasst von Nur Irmayanti und 4 weiteren Autor:innen. Die Arbeit nennt 54 Literaturangaben.
Abstract (Originalsprache)
Early identification of internalizing distress remains challenging because self-report measures may not capture subtle behavioral and affective signals, while nonverbal markers are rarely integrated with linguistic indicators within a clinically interpretable framework. Existing multimodal studies often emphasize predictive accuracy rather than explaining how verbal and facial-affective signals relate to depressive symptoms and broader internalizing processes.
Method
This study developed a clinically informed multimodal conceptual framework integrating linguistic and facial-affective indicators. A mixed-methods design combined PHQ-8 assessments and interview data from 219 participants, linguistic feature extraction targeting uncertainty expressions, negative sentiment, repetition, and discourse variability, facial-affective analysis using FACS-based Action Units and nonverbal behaviors, and expert validation by clinical psychologists and AI specialists. PHQ-8 scores were used specifically as an empirical measure of depressive symptom severity rather than anxiety.
Results
PHQ-8 scores were significantly associated with facial-affective markers (r =.355, p
Zitieren
APA
Irmayanti, N., Ritonga, A. S., Kusnadi, S. K., Wasil, M., & Rahman, S. (2026). A theory-guided multimodal framework for modeling internalizing distress using linguistic and facial-affective indicators. Multidisciplinary Science Journal, 9(4), 2027349. https://doi.org/10.31893/multiscience.2027349
Vancouver
Irmayanti N, Ritonga AS, Kusnadi SK, Wasil M, Rahman S. A theory-guided multimodal framework for modeling internalizing distress using linguistic and facial-affective indicators. Multidisciplinary Science Journal. 2026;9(4):2027349. doi:10.31893/multiscience.2027349
BibTeX
@article{Irmayanti2026,
author = {Irmayanti, Nur and Ritonga, Alven Safik and Kusnadi, Starry Kireida and Wasil, Mohammad and Rahman, Syahnur},
title = {A theory-guided multimodal framework for modeling internalizing distress using linguistic and facial-affective indicators},
journal = {Multidisciplinary Science Journal},
year = {2026},
volume = {9},
number = {4},
pages = {2027349},
doi = {10.31893/multiscience.2027349}
}
EUVIMED