EUVIMEDEUVIMED entdecken
Crossref

Seeing a Talking Face Matters: Gaze Behavior and the Auditory–Visual Speech Benefit in Adults' Cortical Tracking of Infant-directed Speech

Sok Hui Jessica Tan iD, Marina Kalashnikova, Giovanni M Di Liberto, Michael J Crosse, Denis Burnham

Journal of Cognitive Neuroscience · 2023

DOI: 10.1162/jocn_a_02044 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Cognitive Neuroscience, 2023. Verfasst von Sok Hui Jessica Tan und 4 weiteren Autor:innen. Beteiligte Einrichtung: The MARCS Institute of Brain, Behaviour and Development, Western Sydney University, Australia. Hauptthemen laut MEDLINE: Speech, Speech Perception. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 122 Literaturangaben. Gefördert von: The MARCS Institute of Brain, Behaviour and Development, HEARing Cooperative Research Centre, Basque Government.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In face-to-face conversations, listeners gather visual speech information from a speaker's talking face that enhances their perception of the incoming auditory speech signal. This auditory–visual (AV) speech benefit is evident even in quiet environments but is stronger in situations that require greater listening effort such as when the speech signal itself deviates from listeners' expectations. One example is infant-directed speech (IDS) presented to adults. IDS has exaggerated acoustic properties that are easily discriminable from adult-directed speech (ADS). Although IDS is a speech register that adults typically use with infants, no previous neurophysiological study has directly examined whether adult listeners process IDS differently from ADS. To address this, the current study simultaneously recorded EEG and eye-tracking data from adult participants as they were presented with auditory-only (AO), visual-only, and AV recordings of IDS and ADS. Eye-tracking data were recorded because looking behavior to the speaker's eyes and mouth modulates the extent of AV speech benefit experienced. Analyses of cortical tracking accuracy revealed that cortical tracking of the speech envelope was significant in AO and AV modalities for IDS and ADS. However, the AV speech benefit [i.e., AV > (A + V)] was only present for IDS trials. Gaze behavior analyses indicated differences in looking behavior during IDS and ADS trials. Surprisingly, looking behavior to the speaker's eyes and mouth was not correlated with cortical tracking accuracy. Additional exploratory analyses indicated that attention to the whole display was negatively correlated with cortical tracking accuracy of AO and visual-only trials in IDS. Our results underscore the nuances involved in the relationship between neurophysiological AV speech benefit and looking behavior.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SpeechSpeech PerceptionHumansAdultInfantAcoustic StimulationCommunication

Autor:innen und Institutionen

  • The MARCS Institute of Brain, Behaviour and Development, Western Sydney University, Australia.
  • Science of Learning in Education Centre, Office of Education Research, National Institute of Education, Nanyang Technological University, Singapore.
  • The Basque Center on Cognition, Brain and Language.
  • IKERBASQUE, Basque Foundation for Science.
  • ADAPT Centre, School of Computer Science and Statistics, Trinity College Institute of Neuroscience, Trinity College, The University of Dublin, Ireland.
  • SEGOTIA, Galway, Ireland.

ORCID-Profile: Sok Hui Jessica Tan

Förderung

  • The MARCS Institute of Brain, Behaviour and Development
  • HEARing Cooperative Research Centre
  • Basque Government
  • Ministerio de Ciencia e Innovación
  • Science Foundation Ireland

Zitieren

APA

Tan, S. H. J., Kalashnikova, M., Di Liberto, G. M., Crosse, M. J., & Burnham, D. (2023). Seeing a Talking Face Matters: Gaze Behavior and the Auditory–Visual Speech Benefit in Adults' Cortical Tracking of Infant-directed Speech. Journal of Cognitive Neuroscience. https://doi.org/10.1162/jocn_a_02044

Vancouver

Tan SHJ, Kalashnikova M, Di Liberto GM, Crosse MJ, Burnham D. Seeing a Talking Face Matters: Gaze Behavior and the Auditory–Visual Speech Benefit in Adults' Cortical Tracking of Infant-directed Speech. Journal of Cognitive Neuroscience. 2023. doi:10.1162/jocn_a_02044

BibTeX
@article{Tan2023,
  author = {Tan, Sok Hui Jessica and Kalashnikova, Marina and Di Liberto, Giovanni M and Crosse, Michael J and Burnham, Denis},
  title = {Seeing a Talking Face Matters: Gaze Behavior and the Auditory–Visual Speech Benefit in Adults' Cortical Tracking of Infant-directed Speech},
  journal = {Journal of Cognitive Neuroscience},
  year = {2023},
  doi = {10.1162/jocn_a_02044}
}
Kostenlos starten →