EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

NSX-Net: A Neurolinguistic and Acoustic Multimodal Deep Learning Framework for Speech Disorder Classification

Adnan Nadeem iD, Mohammad Zubair Khan iD, Mehreen Sirshar, Usharani Thirunavukkarasu

Diagnostics · 28.07.2026 · 16 (15) · 2377

DOI: 10.3390/diagnostics16152377 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Diagnostics, online erschienen am 28.07.2026. Verfasst von Adnan Nadeem und 3 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Computer and Information Systems, Islamic University of Madinah, Medina 42351, Saudi Arabia. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: King Salman Center for Disability Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Objectives: Speech disorders caused by neurological, articulatory, cognitive, and behavioral impairments require accurate multimodal analysis frameworks capable of jointly understanding acoustic abnormalities and neurolinguistic inconsistencies for reliable computer-assisted speech disorder assessment. Recent multimodal deep learning approaches have utilized speech signals, linguistic transcripts, attention mechanisms, and contextual fusion strategies to improve disordered speech analysis and intelligent disorder prediction. However, existing methods frequently suffer from insufficient temporal synchronization, ineffective local–global feature learning, multimodal redundancy, poor interpretability, and reduced robustness under heterogeneous speech disorders. To address these challenges, Methods: The proposes NSX-Net (NeuroSpeech Explainable Network), an intelligent multimodal deep learning framework for speech disorder classification using Acoustic Speech Signal Data and Neurolinguistic Text/Linguistic Data. Initially, the Adaptive Speech Refinement Module (ASRM) performs noise removal, silence elimination, signal normalization, spectrogram generation, MFCC extraction, and transcript preprocessing to improve multimodal speech consistency. Subsequently, the Hierarchical Multimodal Feature Learning Unit (HMFLU) extracts discriminative feature representations through the Cross-Domain Representation Encoder (CDRE), Fine-Grained Speech Pattern Analyzer (FGSPA), and Global Sequential Dependency Learner (GSDL) for capturing both local articulation abnormalities and long-range semantic dependencies. Furthermore, the Dual-Path Attention Enhancement Block (DPAEB) emphasizes clinically important disordered speech regions using adaptive local–global attention mechanisms, while the Temporal Resolution Synchronization Module (TRSM) aligns rhythm-level and beat-level multimodal contextual structures to improve temporal consistency and suppress noise disturbances.

Results

Experimental evaluation across six publicly available multimodal speech disorder datasets demonstrated that NSX-Net achieved superior performance, with 99.27% Accuracy, 99.11% Precision, 98.97% Recall, 99.02% F1-Score, and 99.41% AUC, significantly outperforming existing state-of-the-art frameworks.

Conclusions

Finally, optimized multimodal contextual representations are forwarded into a Softmax classification layer for intelligent multiclass speech disorder prediction with improved interpretability and potential clinical applicability.

Schlagwörter

Stichwörter der Autor:innen

attention-based deep learningdiagnostics of speech disorderdisordered speech classificationmultimodal speech disorder analysisspeech disorder

Autor:innen und Institutionen

  • Faculty of Computer and Information Systems, Islamic University of Madinah, Medina 42351, Saudi Arabia.
  • King Salman Center for Disability Research, Riyadh 11614, Saudi Arabia.
  • Department of Software Engineering, Fatima Jinnah Women University, Rawalpindi 46000, Punjab, Pakistan.
  • Department of Biomedical Engineering, Saveetha Engineering College, Chennai 602105, India.

ORCID-Profile: Adnan Nadeem, Mohammad Zubair Khan

Förderung

  • King Salman Center for Disability Research

Zitieren

APA

Nadeem, A., Khan, M. Z., Sirshar, M., & Thirunavukkarasu, U. (2026). NSX-Net: A Neurolinguistic and Acoustic Multimodal Deep Learning Framework for Speech Disorder Classification. Diagnostics, 16(15), 2377. https://doi.org/10.3390/diagnostics16152377

Vancouver

Nadeem A, Khan MZ, Sirshar M, Thirunavukkarasu U. NSX-Net: A Neurolinguistic and Acoustic Multimodal Deep Learning Framework for Speech Disorder Classification. Diagnostics. 2026;16(15):2377. doi:10.3390/diagnostics16152377

BibTeX
@article{Nadeem2026,
  author = {Nadeem, Adnan and Khan, Mohammad Zubair and Sirshar, Mehreen and Thirunavukkarasu, Usharani},
  title = {NSX-Net: A Neurolinguistic and Acoustic Multimodal Deep Learning Framework for Speech Disorder Classification},
  journal = {Diagnostics},
  year = {2026},
  volume = {16},
  number = {15},
  pages = {2377},
  doi = {10.3390/diagnostics16152377}
}
Kostenlos starten →