Whisper-RELATE: Natural Speech Representations Predict Reliability-Screened Dynamic Cortical Connectivity Beyond Acoustic and Temporal Baselines
Results in Engineering · 2026-11 · 114084
DOI: 10.1016/j.rineng.2026.114084 ↗Kurz gefasst
Fachartikel in der Zeitschrift Results in Engineering, 2026-11. Verfasst von Mohammad Amin Saket und 1 weiteren Autor:innen. Die Arbeit nennt 76 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Deep speech models predict local neural responses during naturalistic listening, while narrative studies show that functional interactions across cortical networks evolve as a story unfolds. Whether learned representations of continuous speech can predict these evolving interregional interactions remains much less explored. We developed Whisper-RELATE (Reliability-screened Edge Learning with Acoustic controls and Transfer Evaluation), a reliability-aware framework for predicting dynamic cortical edge trajectories from multilevel Whisper representations. Across three cohorts from the public Narratives collection (N=82, 54, and 47), Schaefer-200 signals were converted into time-resolved cortical graphs, and a story-specific response space of 100 reliable dynamic edges was defined by brain-side screening before Whisper modeling. Whisper-based models were compared with matched temporal and low-level acoustic predictors under held-out generalization across time, participants, and dual time-participant splits. Temporal specificity was tested using exhaustive circular-shift surrogates, and the locked procedure was evaluated in two additional narratives without model reselection. All 18 held-out scores were positive, and 17 of 18 exceeded the matched temporal-plus-acoustic baseline, although the magnitude of the incremental benefit varied across validation settings. In all six retained story-model comparisons, the aligned speech representation outperformed every admissible circularly shifted surrogate (empirical p=0.013-0.018). The predictable signal also showed recurrent large-scale organization within the reliability-selected subnetworks, with Somatomotor-Visual and Default-Somatomotor among the leading network axes across stories. These findings show that learned natural-speech representations contain information about evolving interregional cortical interactions beyond measured temporal and low-level acoustic structure when the brain response is reliability-qualified and the mapping is validated for temporal specificity.
Zitieren
APA
Saket, M. A., & Pakravan, M. (2026). Whisper-RELATE: Natural Speech Representations Predict Reliability-Screened Dynamic Cortical Connectivity Beyond Acoustic and Temporal Baselines. Results in Engineering, 114084. https://doi.org/10.1016/j.rineng.2026.114084
Vancouver
Saket MA, Pakravan M. Whisper-RELATE: Natural Speech Representations Predict Reliability-Screened Dynamic Cortical Connectivity Beyond Acoustic and Temporal Baselines. Results in Engineering. 2026:114084. doi:10.1016/j.rineng.2026.114084
BibTeX
@article{Saket2026,
author = {Saket, Mohammad Amin and Pakravan, Mansooreh},
title = {Whisper-RELATE: Natural Speech Representations Predict Reliability-Screened Dynamic Cortical Connectivity Beyond Acoustic and Temporal Baselines},
journal = {Results in Engineering},
year = {2026},
pages = {114084},
doi = {10.1016/j.rineng.2026.114084}
}
EUVIMED