EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A modernized picture description task enables detection of mild cognitive impairment through automated multimodal speech analysis in an open-source pipeline (SPEECH-MARC)

Cynthia Angela Nyongesa, Marcella Antoinette Barneclo, Kimberly Espejo, Yilei Dong, Anna Lisette Isenberg, Judy Pa

Digital Biomarkers · 28.10.2026

DOI: 10.1159/dib/acpag005 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Digital Biomarkers, online erschienen am 28.10.2026. Verfasst von Cynthia Angela Nyongesa und 5 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Mild cognitive impairment (MCI) affects an estimated 15 to 20 percent of adults over 65 and precedes Alzheimer’s disease in up to half of cases within five years, yet remains chronically underdiagnosed in primary care settings where brief, accessible screening tools are most needed. Speech produced during structured picture description tasks offers a non-invasive, scalable signal for early detection, but the field lacks open, reproducible pipelines validated in prospective clinical cohorts. We developed SPEECH-MARC, an open-source multimodal framework that extracts acoustic, linguistic, and scene-level contextual features from picture description speech to classify MoCA-defined cognitive status.

Methods

In a prospective cohort of 127 community-dwelling older adults (cognitively unimpaired, CU: n=61; MCI: n=66) recruited through the UC San Diego Brain Health Virtual Reality Study, we extracted 192 candidate features spanning eGeMAPS acoustic parameters, lexical diversity indices, syntactic complexity measures, and an automated scoring framework for Information Content Units (ICUs) and Areas of Interest (AOIs) adapted for a culturally updated picture description stimulus. Classification used stratified 5-fold cross-validation with feature selection (k=20 features per fold) performed strictly within each training fold, maintaining an effective 1:5 feature-to-training-sample ratio appropriate for small-cohort machine learning.

Results

Combined multimodal models achieved AUC=0.662 (95% CI: 0.563 to 0.755), outperforming acoustic-only (AUC=0.619), linguistic-only (AUC=0.603), and ICU/AOI-only (AUC=0.629) approaches. The strongest predictors were spectral flux variability, lexical diversity, and peripheral scene coverage. Mean word length showed the strongest continuous association with MoCA total score (r=0.31, p=0.0003). Multiple lexical and scene-description features showed FDR-corrected associations with category fluency, digit symbol, and design fluency scores measured independently in the same visit.

Conclusion

Multimodal speech features extracted from a single picture description recording provide clinically interpretable discrimination of MoCA-defined cognitive status. Findings are cross-sectional and require longitudinal validation, but the feature patterns identified are consistent with known profiles of early cognitive decline. The framework is freely available as a Python package with a web-based interface requiring no programming expertise, intended to support research replication and pilot deployment in settings where access to formal neuropsychological testing is limited.

Zitieren

APA

Nyongesa, C. A., Barneclo, M. A., Espejo, K., Dong, Y., Isenberg, A. L., & Pa, J. (2026). A modernized picture description task enables detection of mild cognitive impairment through automated multimodal speech analysis in an open-source pipeline (SPEECH-MARC). Digital Biomarkers. https://doi.org/10.1159/dib/acpag005

Vancouver

Nyongesa CA, Barneclo MA, Espejo K, Dong Y, Isenberg AL, Pa J. A modernized picture description task enables detection of mild cognitive impairment through automated multimodal speech analysis in an open-source pipeline (SPEECH-MARC). Digital Biomarkers. 2026. doi:10.1159/dib/acpag005

BibTeX
@article{Nyongesa2026,
  author = {Nyongesa, Cynthia Angela and Barneclo, Marcella Antoinette and Espejo, Kimberly and Dong, Yilei and Isenberg, Anna Lisette and Pa, Judy},
  title = {A modernized picture description task enables detection of mild cognitive impairment through automated multimodal speech analysis in an open-source pipeline (SPEECH-MARC)},
  journal = {Digital Biomarkers},
  year = {2026},
  doi = {10.1159/dib/acpag005}
}
Kostenlos starten →