EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multimodal processing in simultaneous interpreting with text: Evidence from ear-eye-voice span and performance

Shanshan Yang iD, Defeng Li, Victoria Lai Cheng Lei

PLOS One · 2025

DOI: 10.1371/journal.pone.0326527 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Shanshan Yang und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of English, Central China Normal University, Wuhan, Hubei Province, China. Hauptthemen laut MEDLINE: Speech Perception, Voice, Auditory Perception, Translating. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the Fundamental Research Funds for the Central Universities at Central China Normal University, University of Macau MYRG, National MTI Education and Research Project Fund.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Simultaneous interpreting (SI) with text, a hybrid modality combining auditory and visual inputs, presents greater cognitive complexity than traditional SI. This study investigates multimodal processing in Chinese-English SI with text by examining how source speech rate and professional experience modulate interpreters’ Ear-Eye-Voice Span (EIVS)—a temporal measure reflecting the cognitive coordination among auditory input, visual processing, and verbal output—and interpreting performance. Using eye-tracking technology, we analyzed EIVS patterns in 15 professional interpreters and 30 interpreting trainees performing three SI with text tasks at slow, intermediate and fast speech rates. EIVS measures, including Ear-Eye Span (EIS), Eye-Voice Span (IVS), and Ear-Voice Span (EVS), were analyzed to assess temporal coordination of listening, reading and interpreting processes. Results indicate that faster speech rates significantly reduced EIVS across all measures, suggesting accelerated information processing and strategic cognitive adaptation. A significant interaction effect between speech rate and professional experience was observed. Professionals maintained more stable and efficient EIVS patterns, particularly under accelerated speech rates, reflecting an advantage in cross-modal attention allocation and cognitive resource management. In contrast, trainees exhibited greater reliance on visual input, and struggled more with multimodal demands, manifested in longer EIVS values and greater individual variation. Both groups exhibited an ear-lead-eye coordination pattern during the fast speech rate task, though professionals achieved more efficient auditory-visual synchronization. Despite a decline in interpreting performance with increasing speech rates, professionals consistently outperformed trainees. These findings underscore the critical role of experience in enhancing multimodal coordination, and highlight the importance of dedicated skill-specific practice in enhancing auditory-visual coordination and optimizing interpreting performance under cognitively demanding conditions.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Speech PerceptionVoiceAuditory PerceptionTranslatingHumansMaleAdultFemaleYoung AdultReadingVisual PerceptionEye-Tracking Technology

Autor:innen und Institutionen

  • Department of English, Central China Normal University, Wuhan, Hubei Province, China.
  • Department of English, University of Macau, Macao SAR, China.

ORCID-Profile: Shanshan Yang

Förderung

  • the Fundamental Research Funds for the Central Universities at Central China Normal University
  • University of Macau MYRG
  • National MTI Education and Research Project Fund

Zitieren

APA

Yang, S., Li, D., & Lai Cheng Lei, V. (2025). Multimodal processing in simultaneous interpreting with text: Evidence from ear-eye-voice span and performance. PLOS One. https://doi.org/10.1371/journal.pone.0326527

Vancouver

Yang S, Li D, Lai Cheng Lei V. Multimodal processing in simultaneous interpreting with text: Evidence from ear-eye-voice span and performance. PLOS One. 2025. doi:10.1371/journal.pone.0326527

BibTeX
@article{Yang2025,
  author = {Yang, Shanshan and Li, Defeng and Lai Cheng Lei, Victoria},
  title = {Multimodal processing in simultaneous interpreting with text: Evidence from ear-eye-voice span and performance},
  journal = {PLOS One},
  year = {2025},
  doi = {10.1371/journal.pone.0326527}
}
Kostenlos starten →