EUVIMEDDescobrir EUVIMED
Open access · CC BY-NC 4.0

Perceptual uncertainty explains activation differences between audiovisual congruent speech and McGurk stimuli

Chenjie Dong iD, Uta Noppeney, Suiping Wang

Human Brain Mapping · 2024

DOI: 10.1002/hbm.26653 ↗

At a glance

Journal article in Human Brain Mapping, 2024. Written by Chenjie Dong and 2 co-authors. Institution: Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Educatio…. Major MEDLINE topics: Illusions, Speech Perception. Cited 4 times so far (Crossref); 72 references. The full text is freely available (open access, CC BY-NC 4.0). Funded by: Key Research and Development Program of Guangdong, China, National Natural Science Foundation of China.

ResumoSubjectsDados bibliográficosCiteRelated studies

Abstract (original language)

Face‐to‐face communication relies on the integration of acoustic speech signals with the corresponding facial articulations. In the McGurk illusion, an auditory /ba/ phoneme presented simultaneously with a facial articulation of a /ga/ (i.e., viseme), is typically fused into an illusory ‘da’ percept. Despite its widespread use as an index of audiovisual speech integration, critics argue that it arises from perceptual processes that differ categorically from natural speech recognition. Conversely, Bayesian theoretical frameworks suggest that both the illusory McGurk and the veridical audiovisual congruent speech percepts result from probabilistic inference based on noisy sensory signals. According to these models, the inter‐sensory conflict in McGurk stimuli may only increase observers' perceptual uncertainty. This functional magnetic resonance imaging (fMRI) study presented participants (20 male and 24 female) with audiovisual congruent, McGurk (i.e., auditory /ba/ + visual /ga/), and incongruent (i.e., auditory /ga/ + visual /ba/) stimuli along with their unisensory counterparts in a syllable categorization task. Behaviorally, observers' response entropy was greater for McGurk compared to congruent audiovisual stimuli. At the neural level, McGurk stimuli increased activations in a widespread neural system, extending from the inferior frontal sulci (IFS) to the pre‐supplementary motor area (pre‐SMA) and insulae, typically involved in cognitive control processes. Crucially, in line with Bayesian theories these activation increases were fully accounted for by observers' perceptual uncertainty as measured by their response entropy. Our findings suggest that McGurk and congruent speech processing rely on shared neural mechanisms, thereby supporting the McGurk illusion as a valid measure of natural audiovisual speech perception.

Subjects

MeSH terms (MEDLINE)

IllusionsSpeech PerceptionHumansMaleFemaleAuditory PerceptionSpeechVisual PerceptionBayes TheoremUncertaintyAcoustic StimulationPhotic Stimulation

Author keywords

Bayesian causal inferenceMcGurk illusionfMRIperceptual uncertainty

Authors and institutions

  • Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, China.
  • Donders Institute for Brain, Cognition, and Behavior, Radboud University, Nijmegen, the Netherlands.

ORCID profiles: Chenjie Dong

Funding

  • Key Research and Development Program of Guangdong, China
  • National Natural Science Foundation of China

Cite

APA

Dong, C., Noppeney, U., & Wang, S. (2024). Perceptual uncertainty explains activation differences between audiovisual congruent speech and McGurk stimuli. Human Brain Mapping. https://doi.org/10.1002/hbm.26653

Vancouver

Dong C, Noppeney U, Wang S. Perceptual uncertainty explains activation differences between audiovisual congruent speech and McGurk stimuli. Human Brain Mapping. 2024. doi:10.1002/hbm.26653

BibTeX
@article{Dong2024,
  author = {Dong, Chenjie and Noppeney, Uta and Wang, Suiping},
  title = {Perceptual uncertainty explains activation differences between audiovisual congruent speech and McGurk stimuli},
  journal = {Human Brain Mapping},
  year = {2024},
  doi = {10.1002/hbm.26653}
}
Start for free →