EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Alzheimer’s Dementia Speech (Audio vs. Text): Multi-Modal Machine Learning at High vs. Low Resolution

Prachee Priyadarshinee, Christopher Johann Clarke, Jan Melechovsky, Cindy Ming Ying Lin, Balamurali B. T., Jer-Ming Chen

Applied Sciences · 27.03.2023 · 13 (7) · 4244

DOI: 10.3390/app13074244 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 27.03.2023. Verfasst von Prachee Priyadarshinee und 5 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: SUTD Growth Plan (SGP).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automated techniques to detect Alzheimer’s Dementia through the use of audio recordings of spontaneous speech are now available with varying degrees of reliability. Here, we present a systematic comparison across different modalities, granularities and machine learning models to guide in choosing the most effective tools. Specifically, we present a multi-modal approach (audio and text) for the automatic detection of Alzheimer’s Dementia from recordings of spontaneous speech. Sixteen features, including four feature extraction methods (Energy–Time plots, Keg of Text Analytics, Keg of Text Analytics-Extended and Speech to Silence ratio) not previously applied in this context were tested to determine their relative performance. These features encompass two modalities (audio vs. text) at two resolution scales (frame-level vs. file-level). We compared the accuracy resulting from these features and found that text-based classification outperformed audio-based classification with the best performance attaining 88.7%, surpassing other reports to-date relying on the same dataset. For text-based classification in particular, the best file-level feature performed 9.8% better than the frame-level feature. However, when comparing audio-based classification, the best frame-level feature performed 1.4% better than the best file-level feature. This multi-modal multi-model comparison at high- and low-resolution offers insights into which approach is most efficacious, depending on the sampling context. Such a comparison of the accuracy of Alzheimer’s Dementia classification using both frame-level and file-level granularities on audio and text modalities of different machine learning models on the same dataset has not been previously addressed. We also demonstrate that the subject’s speech captured in short time frames and their dynamics may contain enough inherent information to indicate the presence of dementia. Overall, such a systematic analysis facilitates the identification of Alzheimer’s Dementia quickly and non-invasively, potentially leading to more timely interventions and improved patient outcomes.

Förderung

  • SUTD Growth Plan (SGP)

Zitieren

APA

Priyadarshinee, P., Clarke, C. J., Melechovsky, J., Lin, C. M. Y., T., B. B., & Chen, J. (2023). Alzheimer’s Dementia Speech (Audio vs. Text): Multi-Modal Machine Learning at High vs. Low Resolution. Applied Sciences, 13(7), 4244. https://doi.org/10.3390/app13074244

Vancouver

Priyadarshinee P, Clarke CJ, Melechovsky J, Lin CMY, T. BB, Chen J. Alzheimer’s Dementia Speech (Audio vs. Text): Multi-Modal Machine Learning at High vs. Low Resolution. Applied Sciences. 2023;13(7):4244. doi:10.3390/app13074244

BibTeX
@article{Priyadarshinee2023,
  author = {Priyadarshinee, Prachee and Clarke, Christopher Johann and Melechovsky, Jan and Lin, Cindy Ming Ying and T., Balamurali B. and Chen, Jer-Ming},
  title = {Alzheimer’s Dementia Speech (Audio vs. Text): Multi-Modal Machine Learning at High vs. Low Resolution},
  journal = {Applied Sciences},
  year = {2023},
  volume = {13},
  number = {7},
  pages = {4244},
  doi = {10.3390/app13074244}
}
Kostenlos starten →