Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study
PLOS ONE · 07.10.2024 · 19 (10) · e0306251
DOI: 10.1371/journal.pone.0306251 ↗Kurz gefasst
Mixed-Methods-Studie in der Zeitschrift PLOS ONE, online erschienen am 07.10.2024. Verfasst von Agnieszka Szarkowska und 5 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Applied Linguistics, University of Warsaw, Warsaw, Poland. Hauptthemen laut MEDLINE: Cognition, Eye-Tracking Technology, Comprehension. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 78 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Narodowe Centrum Nauki.
Abstract (Originalsprache)
Every day, millions of viewers worldwide engage with subtitled content, and an increasing number choose to watch without sound. In this mixed-methods study, we examine the impact of sound presence or absence on the viewing experience of both first-language (L1) and second-language (L2) viewers when they watch subtitled videos. We explore this novel phenomenon through comprehension and recall post-tests, self-reported cognitive load, immersion, and enjoyment measures, as well as gaze pattern analysis using eye tracking. We also investigate viewers’ motivations for opting for audiovisual content without sound and explore how the absence of sound impacts their viewing experience, using in-depth, semi-structured interviews. Our goal is to ascertain whether these effects are consistent among L2 and L1 speakers from different language varieties. To achieve this, we tested L1-British English, L1-Australian English and L2-English (L1-Polish) language speakers (n = 168) while they watched English-language audiovisual material with English subtitles with and without sound. The findings show that when watching videos without sound, viewers experienced increased cognitive load, along with reduced comprehension, immersion and overall enjoyment. Examination of participants’ gaze revealed that the absence of sound significantly affected the viewing experience, increasing the need for subtitles and thus increasing the viewers’ propensity to process them more thoroughly. The absence of sound emerged as a global constraint that made reading more effortful. Triangulating data from multiple sources made it possible to tap into some of the metacognitive strategies employed by viewers to maintain comprehension in the absence of sound. We discuss the implications within the context of the growing trend of watching subtitled videos without sound, emphasising its potential impact on cognitive processes and the viewing experience.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Faculty of Applied Linguistics, University of Warsaw, Warsaw, Poland.
- School of Politics, Philosophy, Language and Communication Studies, University of East Anglia, Norwich, United Kingdom.
- School of Modern Languages and Cultures, University of Warwick, Coventry, United Kingdom.
- Department of Linguistics and Language Practice, University of the Free State, Bloemfontein, South Africa.
- Department of Linguistics, Macquarie University, Sydney, Australia.
- UPSET Focus Area, North-West University, Vanderbijlpark, South Africa.
ORCID-Profile: Agnieszka Szarkowska
Förderung
- Narodowe Centrum Nauki
Zitieren
APA
Szarkowska, A., Ragni, V., Szkriba, S., Black, S., Orrego-Carmona, D., & Kruger, J. L. (2024). Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study. PLOS ONE, 19(10), e0306251. https://doi.org/10.1371/journal.pone.0306251
Vancouver
Szarkowska A, Ragni V, Szkriba S, Black S, Orrego-Carmona D, Kruger JL. Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study. PLOS ONE. 2024;19(10):e0306251. doi:10.1371/journal.pone.0306251
BibTeX
@article{Szarkowska2024,
author = {Szarkowska, Agnieszka and Ragni, Valentina and Szkriba, Sonia and Black, Sharon and Orrego-Carmona, David and Kruger, Jan-Louis},
title = {Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study},
journal = {PLOS ONE},
year = {2024},
volume = {19},
number = {10},
pages = {e0306251},
doi = {10.1371/journal.pone.0306251}
}
EUVIMED