EUVIMEDEUVIMED entdecken
Übersichtsarbeit

Relating EEG to continuous speech using deep neural networks: a review

Corentin Puffay iD, Bernd Accou iD, Lies Bollens iD, Mohammad Jalilpour Monesi iD, Jonas Vanthornhout iD, Hugo Van Hamme iD, Tom Francart iD

Journal of Neural Engineering · 01.08.2023 · 20 (4) · 041003

DOI: 10.1088/1741-2552/ace73f ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift Journal of Neural Engineering, online erschienen am 03.08.2023. Verfasst von Corentin Puffay und 6 weiteren Autor:innen. Beteiligte Einrichtung: KU Leuven, Department of Neurosciences, ExpORL, Leuven, Belgium. Hauptthemen laut MEDLINE: Speech, Electroencephalography. Die Arbeit wurde bisher 88-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: Fonds Wetenschappelijk Onderzoek, KU Leuven.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective. When a person listens to continuous speech, a corresponding response is elicited in the brain and can be recorded using electroencephalography (EEG). Linear models are presently used to relate the EEG recording to the corresponding speech signal. The ability of linear models to find a mapping between these two signals is used as a measure of neural tracking of speech. Such models are limited as they assume linearity in the EEG-speech relationship, which omits the nonlinear dynamics of the brain. As an alternative, deep learning models have recently been used to relate EEG to continuous speech. Approach. This paper reviews and comments on deep-learning-based studies that relate EEG to continuous speech in single- or multiple-speakers paradigms. We point out recurrent methodological pitfalls and the need for a standard benchmark of model analysis. Main results. We gathered 29 studies. The main methodological issues we found are biased cross-validations, data leakage leading to over-fitted models, or disproportionate data size compared to the model’s complexity. In addition, we address requirements for a standard benchmark model analysis, such as public datasets, common evaluation metrics, and good practices for the match-mismatch task. Significance. We present a review paper summarizing the main deep-learning-based studies that relate EEG to speech while addressing methodological pitfalls and important considerations for this newly expanding field. Our study is particularly relevant given the growing application of deep learning in EEG-speech decoding.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SpeechElectroencephalographyHumansNeural Networks, ComputerBrainAuditory Perception

Stichwörter der Autor:innen

EEGauditory neurosciencedeep learningspeech

Autor:innen und Institutionen

  • KU Leuven, Department of Neurosciences, ExpORL, Leuven, Belgium.
  • KU Leuven, Department of Electrical Engineering (ESAT), PSI, Leuven, Belgium.

ORCID-Profile: Corentin Puffay, Bernd Accou, Lies Bollens, Mohammad Jalilpour Monesi, Jonas Vanthornhout, Hugo Van Hamme, Tom Francart

Förderung

  • Fonds Wetenschappelijk Onderzoek
  • KU Leuven

Zitieren

APA

Puffay, C., Accou, B., Bollens, L., Monesi, M. J., Vanthornhout, J., Van Hamme, H., & Francart, T. (2023). Relating EEG to continuous speech using deep neural networks: a review. Journal of Neural Engineering, 20(4), 041003. https://doi.org/10.1088/1741-2552/ace73f

Vancouver

Puffay C, Accou B, Bollens L, Monesi MJ, Vanthornhout J, Van Hamme H, et al. Relating EEG to continuous speech using deep neural networks: a review. Journal of Neural Engineering. 2023;20(4):041003. doi:10.1088/1741-2552/ace73f

BibTeX
@article{Puffay2023,
  author = {Puffay, Corentin and Accou, Bernd and Bollens, Lies and Monesi, Mohammad Jalilpour and Vanthornhout, Jonas and Van Hamme, Hugo and Francart, Tom},
  title = {Relating EEG to continuous speech using deep neural networks: a review},
  journal = {Journal of Neural Engineering},
  year = {2023},
  volume = {20},
  number = {4},
  pages = {041003},
  doi = {10.1088/1741-2552/ace73f}
}
Kostenlos starten →