Prior expectations guide multisensory integration during face-to-face communication
PLOS Computational Biology · 2025
DOI: 10.1371/journal.pcbi.1013468 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, 2025. Verfasst von Giulia Mazzi und 6 weiteren Autor:innen. Beteiligte Einrichtung: Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, Italy. Hauptthemen laut MEDLINE: Communication. Die Arbeit nennt 105 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: University of Trento, Velux Foundation, Italian Ministry of Education, University and Research.
Abstract (Originalsprache)
Face-to-face communication relies on the seamless integration of multisensory signals, including voice, gaze, and head movements, to convey meaning effectively. This poses a fundamental computational challenge: optimally binding signals sharing the same communicative intention (e.g., looking at the addressee while speaking) and segregating unrelated signals (e.g., looking away while coughing), all within the rapid turn-taking dynamics of conversation. Critically, the computational mechanisms underlying this extraordinary feat remain largely unknown. Here, we cast face-to-face communication as a Bayesian Causal Inference problem to formally test whether prior expectations arbitrate between the integration and segregation of vocal and bodily signals. Specifically, we asked whether there is a stronger prior tendency to integrate audiovisual signals that convey the same communicative intention, thus establishing a crossmodal pragmatic correspondence. Additionally, we evaluated whether observers solve causal inference by adopting optimal Bayesian decision strategies or non-optimal approximate heuristics. In a spatial localization task, participants watched audiovisual clips of a speaker where the audio (voice) and the video (bodily cues) were sampled either from congruent positions or at increasing spatial disparities. Crucially, we manipulated the pragmatic correspondence of the signals: in a communicative condition, the speaker addressed the participant with their head, gaze and speech; in a non-communicative condition, the speaker kept the head down and produced a meaningless vocalization. We measured audiovisual integration through the ventriloquist effect, which quantifies how much the perceived audio position is misplaced towards the video position. Combining psychophysics with computational modelling, we show that observers solved audiovisual causal inference using non-optimal heuristics that nevertheless approximate optimal Bayesian inference with high accuracy. Remarkably, participants showed a stronger tendency to integrate vocal and bodily information when signals conveyed congruent communicative intent, suggesting that pragmatic correspondences enhance multisensory integration. Collectively, our findings provide novel and compelling evidence that face-to-face communication is shaped by deeply ingrained expectations about how multisensory signals should be structured and interpreted.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, Italy.
- Max Plank Institute for Psycholinguistics, Nijmegen, The Netherlands.
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands.
- Department of Psychology and Cognitive Sciences, University of Trento, Rovereto, Italy.
- Centro Interuniversitario di Ricerca "Cognizione, Linguaggio e Sordità", CIRCLeS., Trento, Italy.
ORCID-Profile: Giulia Mazzi, Chiara Valzolgher
Förderung
- University of Trento
- Velux Foundation
- Italian Ministry of Education, University and Research
- Università degli Studi di Trento
- Ministero dell’Istruzione, dell’Università e della Ricerca
Zitieren
APA
Mazzi, G., Ferrari, A., Mencaroni, M. L., Valzolgher, C., Tommasini, M., Pavani, F., & Benetti, S. (2025). Prior expectations guide multisensory integration during face-to-face communication. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1013468
Vancouver
Mazzi G, Ferrari A, Mencaroni ML, Valzolgher C, Tommasini M, Pavani F, et al. Prior expectations guide multisensory integration during face-to-face communication. PLOS Computational Biology. 2025. doi:10.1371/journal.pcbi.1013468
BibTeX
@article{Mazzi2025,
author = {Mazzi, Giulia and Ferrari, Ambra and Mencaroni, Maria Laura and Valzolgher, Chiara and Tommasini, Mirko and Pavani, Francesco and Benetti, Stefania},
title = {Prior expectations guide multisensory integration during face-to-face communication},
journal = {PLOS Computational Biology},
year = {2025},
doi = {10.1371/journal.pcbi.1013468}
}
EUVIMED