EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The effect of speech–gesture asynchrony on the neural coupling of interlocutors in interpreter-mediated communication

Xu Duan iD, Jie Zhang, Yi Zhang, Yuan Liang, Yingying Huang, Hao Yan

Social Cognitive and Affective Neuroscience · 2023

DOI: 10.1093/scan/nsad027 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Social Cognitive and Affective Neuroscience, 2023. Verfasst von Xu Duan und 5 weiteren Autor:innen. Beteiligte Einrichtung: Key Laboratory for Artificial Intelligence and Cognitive Neuroscience of Language, Xi'an International Studies University, No. 6 Wenyuan South Road, …. Hauptthemen laut MEDLINE: Speech, Gestures. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Social Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In everyday face-to-face communication, speakers use speech to transfer information and rely on co-occurring nonverbal cues, such as hand and facial gestures. The integration of speech and gestures facilitates both language comprehension and the skill of the theory of mind. Consecutive dialogue interpreting (DI) allows dyads of different linguistic backgrounds to communicate with each other. The interpreter interprets after the interlocutor has finished a turn, so the interlocutor watches the gesture first and hears the target language a few seconds later, resulting in speech–gesture asynchrony. In this study, we used the functional near-infrared spectroscopy hyperscanning technique to investigate the influence of speech–gesture asynchrony on different levels of communication. Twenty groups were recruited for the DI experiments. The results showed that when the interpreter performed consecutive interpreting, the time-lagged neural coupling at the temporoparietal junction decreased compared to simultaneous interpreting. It suggests that speech–gesture asynchrony significantly weakened the ability of interlocutors to understand each other’s mental state, and the decreased neural coupling was significantly correlated with the interpreter’s interpretation skill. In addition, the time-aligned neural coupling at the left inferior frontal gyrus increased, which suggests that, as compensation, the interlocutor’s verbal working memory increases in line with the communication process.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SpeechGesturesHumansCommunicationLanguageMemoryCuesComprehension

Stichwörter der Autor:innen

dialogue interpretingfNIRS hyperscanningneural couplingspeech–gesture asynchronyverbal communication

Autor:innen und Institutionen

  • Key Laboratory for Artificial Intelligence and Cognitive Neuroscience of Language, Xi'an International Studies University, No. 6 Wenyuan South Road, Xi'an, Shannxi 710128, P.R. China.
  • Department of Radiation Medicine, Air Force Military Medical University, No. 169 Changle West Road, Xi'an, Shannxi 710032, P.R. China.
  • Department of Linguistics, Xidian University, No. 2 Taibai South Road, Xi'an, Shannxi 710071, P.R. China.

ORCID-Profile: Xu Duan

Förderung

  • National Social Science Foundation of China

Zitieren

APA

Duan, X., Zhang, J., Zhang, Y., Liang, Y., Huang, Y., & Yan, H. (2023). The effect of speech–gesture asynchrony on the neural coupling of interlocutors in interpreter-mediated communication. Social Cognitive and Affective Neuroscience. https://doi.org/10.1093/scan/nsad027

Vancouver

Duan X, Zhang J, Zhang Y, Liang Y, Huang Y, Yan H. The effect of speech–gesture asynchrony on the neural coupling of interlocutors in interpreter-mediated communication. Social Cognitive and Affective Neuroscience. 2023. doi:10.1093/scan/nsad027

BibTeX
@article{Duan2023,
  author = {Duan, Xu and Zhang, Jie and Zhang, Yi and Liang, Yuan and Huang, Yingying and Yan, Hao},
  title = {The effect of speech–gesture asynchrony on the neural coupling of interlocutors in interpreter-mediated communication},
  journal = {Social Cognitive and Affective Neuroscience},
  year = {2023},
  doi = {10.1093/scan/nsad027}
}
Kostenlos starten →