Mild Cognitive Impairment Detection System Based on Unstructured Spontaneous Speech: Longitudinal Dual-Modal Framework
JMIR Medical Informatics · 2026
DOI: 10.2196/80883 ↗At a glance
Journal article in JMIR Medical Informatics, 2026. Written by Yu-Shan Liao and 5 co-authors. Institution: Graduate Institute of Networking and Multimedia, National Taiwan University, Taipei, Taiwan. Major MEDLINE topics: Cognitive Dysfunction, Speech. 33 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
In recent years, the incidence of cognitive diseases has also risen with the significant increase in population aging. Among these diseases, Alzheimer disease constitutes a substantial proportion, placing a high-cost burden on health care systems. To give early treatment and slow the progression of patient deterioration, it is crucial to diagnose mild cognitive impairment (MCI), a transitional stage.
Objective
In this study, we use autobiographical memory (AM) test speech data to establish a dual-modal longitudinal cognitive detection system for MCI. The AM test is a psychological assessment method that evaluates the cognitive status of subjects as they freely narrate important life experiences.
Methods
Identifying hidden disease-related information in unstructured, spontaneous speech is more difficult than in structured speech. To improve this process, we use both speech and text data, which provide more clues about a person’s cognitive state. In addition, to track how cognition changes over time in spontaneous speech, we introduce an aging trajectory module. This module uses local and global alignment loss functions to better learn time-related features by aligning cognitive changes across different time points.
Results
In our experiments on the Chinese dataset, the longitudinal model incorporating the aging trajectory module achieved area under the receiver operating characteristic curve of 0.85 and 0.89 on 2 datasets, respectively, showing significant improvement over cross-sectional, single time point models. We also conducted ablation studies to verify the necessity of the proposed aging trajectory module. To confirm that the model not only applies to AM test data, we used part of the model to evaluate the performance on the ADReSSo dataset, a single time point semistructured data for validation, with results showing an accuracy exceeding 0.88.
Conclusions
This study presents a noninvasive and scalable approach for early MCI detection by leveraging AM speech data across multiple time points. Through dual-modal analysis and the introduction of an aging trajectory module, our system effectively captures cognitive decline trends over time. Experimental results demonstrate the method’s robustness and generalizability, highlighting its potential for real-world, long-term cognitive monitoring.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Graduate Institute of Networking and Multimedia, National Taiwan University, Taipei, Taiwan.
- Department of Computer Science and Information Engineering, National Taiwan University, No. 1, Sec 4, Roosevelt Rd, Taipei, Taiwan, 886 0233663366.
- Department of Psychology, National Taiwan University, Taipei, Taiwan.
- Center of Artificial intelligence Research, University of Tsukuba, Tsukuba, Ibaraki, Japan.
ORCID profiles: Yu-Shan Liao, Thiri Wai, Ting-Yun Liao, Ho-Ling Chang, Yu-Ling Chang, Li-Chen Fu
Cite
APA
Liao, Y. S., Wai, T., Liao, T. Y., Chang, H. L., Chang, Y. L., & Fu, L. C. (2026). Mild Cognitive Impairment Detection System Based on Unstructured Spontaneous Speech: Longitudinal Dual-Modal Framework. JMIR Medical Informatics. https://doi.org/10.2196/80883
Vancouver
Liao YS, Wai T, Liao TY, Chang HL, Chang YL, Fu LC. Mild Cognitive Impairment Detection System Based on Unstructured Spontaneous Speech: Longitudinal Dual-Modal Framework. JMIR Medical Informatics. 2026. doi:10.2196/80883
BibTeX
@article{Liao2026,
author = {Liao, Yu-Shan and Wai, Thiri and Liao, Ting-Yun and Chang, Ho-Ling and Chang, Yu-Ling and Fu, Li-Chen},
title = {Mild Cognitive Impairment Detection System Based on Unstructured Spontaneous Speech: Longitudinal Dual-Modal Framework},
journal = {JMIR Medical Informatics},
year = {2026},
doi = {10.2196/80883}
}
EUVIMED