Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions
PLOS Biology · 13.12.2023 · 21 (12) · e3002366
DOI: 10.1371/journal.pbio.3002366 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Biology, online erschienen am 13.12.2023. Verfasst von Greta Tuckute und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research MIT, Cambridge, Massachusetts, United States of America. Hauptthemen laut MEDLINE: Neural Networks, Computer, Auditory Cortex. Die Arbeit wurde bisher 66-mal zitiert (Crossref) und nennt 172 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIDCD NIH HHS, NIDCD, Department of Brain and Cognitive Sciences, MIT.
Abstract (Originalsprache)
Models that predict brain responses to stimuli provide one measure of understanding of a sensory system and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models and, thus, how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on 4 different tasks. Most tested models outpredicted standard spectromporal filter-bank models of auditory cortex and exhibited systematic model-brain correspondence: Middle stages best predicted primary auditory cortex, while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research MIT, Cambridge, Massachusetts, United States of America.
- Center for Brains, Minds, and Machines, MIT, Cambridge, Massachusetts, United States of America.
- Program in Speech and Hearing Biosciences and Technology, Harvard, Cambridge, Massachusetts, United States of America.
- University of Rochester Medical Center, Rochester, New York, New York, United States of America.
ORCID-Profile: Josh H McDermott
Förderung
- NIDCD NIH HHS
- NIDCD
- Department of Brain and Cognitive Sciences, MIT
- Amazon
- American Association of University Women
- US Department of Energy
- McGovern Institute for Brain Research, Massachusetts Institute of Technology
Zitieren
APA
Tuckute, G., Feather, J., Boebinger, D., & McDermott, J. H. (2023). Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions. PLOS Biology, 21(12), e3002366. https://doi.org/10.1371/journal.pbio.3002366
Vancouver
Tuckute G, Feather J, Boebinger D, McDermott JH. Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions. PLOS Biology. 2023;21(12):e3002366. doi:10.1371/journal.pbio.3002366
BibTeX
@article{Tuckute2023,
author = {Tuckute, Greta and Feather, Jenelle and Boebinger, Dana and McDermott, Josh H},
title = {Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions},
journal = {PLOS Biology},
year = {2023},
volume = {21},
number = {12},
pages = {e3002366},
doi = {10.1371/journal.pbio.3002366}
}
EUVIMED