EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals

Ruixiang Liu, Chang Liu, Dan Cui, Huan Zhang, Xinmeng Xu, Yuxin Duan, Yihu Chao, Xianzheng Sha, Limin Sun, Xiulan Ma, Shuo Li, Shijie Chang iD

Trends in Hearing · 2024

DOI: 10.1177/23312165241282872 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Trends in Hearing, 2024. Verfasst von Ruixiang Liu und 11 weiteren Autor:innen. Beteiligte Einrichtung: School of Intelligent Medicine, China Medical University, Shenyang, China. Hauptthemen laut MEDLINE: Electroencephalography, Speech Perception, Signal Processing, Computer-Assisted, Deep Learning. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Key Research and Development Program of China, Natural Science Foundation of Liaoning Province, General Research Program of Liaoning Provincial Department of Education.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Decoding speech envelopes from electroencephalogram (EEG) signals holds potential as a research tool for objectively assessing auditory processing, which could contribute to future developments in hearing loss diagnosis. However, current methods struggle to meet both high accuracy and interpretability. We propose a deep learning model called the auditory decoding transformer (ADT) network for speech envelope reconstruction from EEG signals to address these issues. The ADT network uses spatio-temporal convolution for feature extraction, followed by a transformer decoder to decode the speech envelopes. Through anticausal masking, the ADT considers only the current and future EEG features to match the natural relationship of speech and EEG. Performance evaluation shows that the ADT network achieves average reconstruction scores of 0.168 and 0.167 on the SparrKULee and DTU datasets, respectively, rivaling those of other nonlinear models. Furthermore, by visualizing the weights of the spatio-temporal convolution layer as time-domain filters and brain topographies, combined with an ablation study of the temporal convolution kernels, we analyze the behavioral patterns of the ADT network in decoding speech envelopes. The results indicate that low- (0.5–8 Hz) and high-frequency (14–32 Hz) EEG signals are more critical for envelope reconstruction and that the active brain regions are primarily distributed bilaterally in the auditory cortex, consistent with previous research. Visualization of attention scores further validated previous research. In summary, the ADT network balances high performance and interpretability, making it a promising tool for studying neural speech envelope tracking.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ElectroencephalographySpeech PerceptionSignal Processing, Computer-AssistedDeep LearningHumansNonlinear DynamicsAcoustic StimulationSpeech AcousticsNeural Networks, ComputerAuditory Cortex

Stichwörter der Autor:innen

deep learningelectroencephalograminterpretabilityneural envelope trackingtransformer

Autor:innen und Institutionen

  • School of Intelligent Medicine, China Medical University, Shenyang, China.
  • Shengjing Hospital of China Medical University, Shenyang, China.
  • Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai, China.
  • School of Life Sciences, China Medical University, Shenyang, China.

ORCID-Profile: Shijie Chang

Förderung

  • National Key Research and Development Program of China
  • Natural Science Foundation of Liaoning Province
  • General Research Program of Liaoning Provincial Department of Education

Zitieren

APA

Liu, R., Liu, C., Cui, D., Zhang, H., Xu, X., Duan, Y., Chao, Y., Sha, X., Sun, L., Ma, X., Li, S., & Chang, S. (2024). ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals. Trends in Hearing. https://doi.org/10.1177/23312165241282872

Vancouver

Liu R, Liu C, Cui D, Zhang H, Xu X, Duan Y, et al. ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals. Trends in Hearing. 2024. doi:10.1177/23312165241282872

BibTeX
@article{Liu2024,
  author = {Liu, Ruixiang and Liu, Chang and Cui, Dan and Zhang, Huan and Xu, Xinmeng and Duan, Yuxin and Chao, Yihu and Sha, Xianzheng and Sun, Limin and Ma, Xiulan and Li, Shuo and Chang, Shijie},
  title = {ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals},
  journal = {Trends in Hearing},
  year = {2024},
  doi = {10.1177/23312165241282872}
}
Kostenlos starten →