ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals
Trends in Hearing · 2024
DOI: 10.1177/23312165241282872 ↗Kurz gefasst
Fachartikel in der Zeitschrift Trends in Hearing, 2024. Verfasst von Ruixiang Liu und 11 weiteren Autor:innen. Beteiligte Einrichtung: School of Intelligent Medicine, China Medical University, Shenyang, China. Hauptthemen laut MEDLINE: Electroencephalography, Speech Perception, Signal Processing, Computer-Assisted, Deep Learning. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Key Research and Development Program of China, Natural Science Foundation of Liaoning Province, General Research Program of Liaoning Provincial Department of Education.
Abstract (Originalsprache)
Decoding speech envelopes from electroencephalogram (EEG) signals holds potential as a research tool for objectively assessing auditory processing, which could contribute to future developments in hearing loss diagnosis. However, current methods struggle to meet both high accuracy and interpretability. We propose a deep learning model called the auditory decoding transformer (ADT) network for speech envelope reconstruction from EEG signals to address these issues. The ADT network uses spatio-temporal convolution for feature extraction, followed by a transformer decoder to decode the speech envelopes. Through anticausal masking, the ADT considers only the current and future EEG features to match the natural relationship of speech and EEG. Performance evaluation shows that the ADT network achieves average reconstruction scores of 0.168 and 0.167 on the SparrKULee and DTU datasets, respectively, rivaling those of other nonlinear models. Furthermore, by visualizing the weights of the spatio-temporal convolution layer as time-domain filters and brain topographies, combined with an ablation study of the temporal convolution kernels, we analyze the behavioral patterns of the ADT network in decoding speech envelopes. The results indicate that low- (0.5–8 Hz) and high-frequency (14–32 Hz) EEG signals are more critical for envelope reconstruction and that the active brain regions are primarily distributed bilaterally in the auditory cortex, consistent with previous research. Visualization of attention scores further validated previous research. In summary, the ADT network balances high performance and interpretability, making it a promising tool for studying neural speech envelope tracking.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Intelligent Medicine, China Medical University, Shenyang, China.
- Shengjing Hospital of China Medical University, Shenyang, China.
- Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai, China.
- School of Life Sciences, China Medical University, Shenyang, China.
ORCID-Profile: Shijie Chang
Förderung
- National Key Research and Development Program of China
- Natural Science Foundation of Liaoning Province
- General Research Program of Liaoning Provincial Department of Education
Zitieren
APA
Liu, R., Liu, C., Cui, D., Zhang, H., Xu, X., Duan, Y., Chao, Y., Sha, X., Sun, L., Ma, X., Li, S., & Chang, S. (2024). ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals. Trends in Hearing. https://doi.org/10.1177/23312165241282872
Vancouver
Liu R, Liu C, Cui D, Zhang H, Xu X, Duan Y, et al. ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals. Trends in Hearing. 2024. doi:10.1177/23312165241282872
BibTeX
@article{Liu2024,
author = {Liu, Ruixiang and Liu, Chang and Cui, Dan and Zhang, Huan and Xu, Xinmeng and Duan, Yuxin and Chao, Yihu and Sha, Xianzheng and Sun, Limin and Ma, Xiulan and Li, Shuo and Chang, Shijie},
title = {ADT Network: A Novel Nonlinear Method for Decoding Speech Envelopes From EEG Signals},
journal = {Trends in Hearing},
year = {2024},
doi = {10.1177/23312165241282872}
}
EUVIMED