Understanding discourse in face-to-face settings: The impact of multimodal cues and listening conditions.
Journal of Experimental Psychology: Learning, Memory, and Cognition · 2025-05 · 51 (5) · 837-854
DOI: 10.1037/xlm0001399 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Experimental Psychology: Learning, Memory, and Cognition, 2025-05. Verfasst von Anna Krason und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Experimental Psychology, University College London. Hauptthemen laut MEDLINE: Gestures, Cues, Comprehension, Speech Perception. Die Arbeit wurde bisher 10-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Councils UK, European Research Council; H2020, The Royal Society.
Abstract (Originalsprache)
In face-to-face contexts, discourse is accompanied by various cues, like gestures and mouth movements. Here, we asked whether the presence of gestures and mouth movements benefits discourse comprehension under clear and challenging listening conditions and, if so, whether this multimodal benefit depends on the communicative environment in which interlocutors are situated. In two online experiments, participants watched videoclips of a speaker telling stories, and they answered yes-no questions about the content of each story. The speaker in the videos was spontaneously gesturing (or kept her hands still) and was wearing a surgical mask (or had her lips visible). The experiments differed in the communicative environment. In Experiment 1, the speaker narrated stories in silence, whereas the listener (participants) heard them in clear or degraded speech conditions (analogous to watching the news on TV in a quiet or noisy café). In Experiment 2, the speaker narrated the stories once in silence and once while listening to background noise, and the listener heard them in clear or degraded speech condition, respectively (analogous to listening to a friend in a quiet or noisy café). Across the experiments, we found that cospeech gestures facilitated discourse comprehension regardless of the listening conditions or the presence of a mask. In contrast, mouth movements were primarily helpful in challenging listening conditions. These findings indicate that both cues matter to listeners but to a different extent. Moreover, we found that the multimodal benefit to comprehension was similar regardless of the communicative environment. Thus, this study demonstrates the importance of both cospeech gestures and mouth movements to discourse comprehension, offering insights into the dynamic interplay between these cues under different communicative environments. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Experimental Psychology, University College London.
- Department of Language and Cognition, University College London.
ORCID-Profile: Anna Krason
Förderung
- Research Councils UK
- European Research Council; H2020
- The Royal Society
- University College London
- Wellcome Trust
- European Research Council
Zitieren
APA
Krason, A., Varley, R., & Vigliocco, G. (2025). Understanding discourse in face-to-face settings: The impact of multimodal cues and listening conditions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 51(5), 837-854. https://doi.org/10.1037/xlm0001399
Vancouver
Krason A, Varley R, Vigliocco G. Understanding discourse in face-to-face settings: The impact of multimodal cues and listening conditions. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2025;51(5):837-854. doi:10.1037/xlm0001399
BibTeX
@article{Krason2025,
author = {Krason, Anna and Varley, Rosemary and Vigliocco, Gabriella},
title = {Understanding discourse in face-to-face settings: The impact of multimodal cues and listening conditions},
journal = {Journal of Experimental Psychology: Learning, Memory, and Cognition},
year = {2025},
volume = {51},
number = {5},
pages = {837--854},
doi = {10.1037/xlm0001399}
}
EUVIMED