A Multimodal CNN Transformer Framework for Early Dyslexia and Dysgraphia Detection Using Handwriting and Speech
International Journal of Innovative Science and Research Technology · 06.05.2026 · 3426
DOI: 10.38124/ijisrt/26apr2145 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Innovative Science and Research Technology, online erschienen am 06.05.2026. Verfasst von Siddhesh Pote und 3 weiteren Autor:innen. Die Arbeit nennt 11 Literaturangaben.
Abstract (Originalsprache)
This paper presents a multimodal framework for early screening of learning disabilities—focusing on dyslexia and dysgraphia—by jointly modeling handwriting and speech to capture graphophonological interactions that singlemodality systems often miss. Prior studies have shown high within-cohort accuracy using CNNs for handwriting, classical machine learning and deep models for EEG and imaging, and educational analytics; however, most rely on small, homogeneous datasets, use late fusion when combining modalities, and lack cross-site or cross-language validation, limiting generalizability and deployment potential. The proposed system integrates a vision encoder for handwriting (optionally incorporating tablet kinematics) and a speech encoder that fuses acoustic and ASR-derived linguistic features via cross-modal transformers, trained with supervised and contrastive losses for robust alignment. Methodological considerations include multilingual data collection, standardized preprocessing, calibrated uncertainty, and privacypreserving learning to support equitable classroom deployment. The evaluation plan compares unimodal baselines, latefusion ensembles, and the proposed intermediate-fusion architecture across within-site, cross-site, and cross-language settings using AUROC, macro-F1, severity kappa, and fairness audits. Expected outcomes include improved out-ofdistribution performance and interpretable per-modality rationales to assist educators and clinicians in early intervention.
Zitieren
APA
Pote, S., Zagade, G., Suryavanshi, S., & Shahapurkar, A. (2026). A Multimodal CNN Transformer Framework for Early Dyslexia and Dysgraphia Detection Using Handwriting and Speech. International Journal of Innovative Science and Research Technology, 3426. https://doi.org/10.38124/ijisrt/26apr2145
Vancouver
Pote S, Zagade G, Suryavanshi S, Shahapurkar A. A Multimodal CNN Transformer Framework for Early Dyslexia and Dysgraphia Detection Using Handwriting and Speech. International Journal of Innovative Science and Research Technology. 2026:3426. doi:10.38124/ijisrt/26apr2145
BibTeX
@article{Pote2026,
author = {Pote, Siddhesh and Zagade, Gaurav and Suryavanshi, Shrikrushna and Shahapurkar, Ashwini},
title = {A Multimodal CNN Transformer Framework for Early Dyslexia and Dysgraphia Detection Using Handwriting and Speech},
journal = {International Journal of Innovative Science and Research Technology},
year = {2026},
pages = {3426},
doi = {10.38124/ijisrt/26apr2145}
}
EUVIMED