EUVIMEDEUVIMED entdecken
Mixed-Methods-StudieOpen Access · CC BY 4.0

Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study

Agnieszka Szarkowska iD, Valentina Ragni, Sonia Szkriba, Sharon Black, David Orrego-Carmona, Jan-Louis Kruger

PLOS ONE · 07.10.2024 · 19 (10) · e0306251

DOI: 10.1371/journal.pone.0306251 ↗

Kurz gefasst

Mixed-Methods-Studie in der Zeitschrift PLOS ONE, online erschienen am 07.10.2024. Verfasst von Agnieszka Szarkowska und 5 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Applied Linguistics, University of Warsaw, Warsaw, Poland. Hauptthemen laut MEDLINE: Cognition, Eye-Tracking Technology, Comprehension. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 78 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Narodowe Centrum Nauki.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Every day, millions of viewers worldwide engage with subtitled content, and an increasing number choose to watch without sound. In this mixed-methods study, we examine the impact of sound presence or absence on the viewing experience of both first-language (L1) and second-language (L2) viewers when they watch subtitled videos. We explore this novel phenomenon through comprehension and recall post-tests, self-reported cognitive load, immersion, and enjoyment measures, as well as gaze pattern analysis using eye tracking. We also investigate viewers’ motivations for opting for audiovisual content without sound and explore how the absence of sound impacts their viewing experience, using in-depth, semi-structured interviews. Our goal is to ascertain whether these effects are consistent among L2 and L1 speakers from different language varieties. To achieve this, we tested L1-British English, L1-Australian English and L2-English (L1-Polish) language speakers (n = 168) while they watched English-language audiovisual material with English subtitles with and without sound. The findings show that when watching videos without sound, viewers experienced increased cognitive load, along with reduced comprehension, immersion and overall enjoyment. Examination of participants’ gaze revealed that the absence of sound significantly affected the viewing experience, increasing the need for subtitles and thus increasing the viewers’ propensity to process them more thoroughly. The absence of sound emerged as a global constraint that made reading more effortful. Triangulating data from multiple sources made it possible to tap into some of the metacognitive strategies employed by viewers to maintain comprehension in the absence of sound. We discuss the implications within the context of the growing trend of watching subtitled videos without sound, emphasising its potential impact on cognitive processes and the viewing experience.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

CognitionEye-Tracking TechnologyComprehensionHumansFemaleMaleAdultYoung AdultVideo RecordingLanguageAdolescentFixation, OcularSoundPleasureMiddle Aged

Autor:innen und Institutionen

  • Faculty of Applied Linguistics, University of Warsaw, Warsaw, Poland.
  • School of Politics, Philosophy, Language and Communication Studies, University of East Anglia, Norwich, United Kingdom.
  • School of Modern Languages and Cultures, University of Warwick, Coventry, United Kingdom.
  • Department of Linguistics and Language Practice, University of the Free State, Bloemfontein, South Africa.
  • Department of Linguistics, Macquarie University, Sydney, Australia.
  • UPSET Focus Area, North-West University, Vanderbijlpark, South Africa.

ORCID-Profile: Agnieszka Szarkowska

Förderung

  • Narodowe Centrum Nauki

Zitieren

APA

Szarkowska, A., Ragni, V., Szkriba, S., Black, S., Orrego-Carmona, D., & Kruger, J. L. (2024). Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study. PLOS ONE, 19(10), e0306251. https://doi.org/10.1371/journal.pone.0306251

Vancouver

Szarkowska A, Ragni V, Szkriba S, Black S, Orrego-Carmona D, Kruger JL. Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study. PLOS ONE. 2024;19(10):e0306251. doi:10.1371/journal.pone.0306251

BibTeX
@article{Szarkowska2024,
  author = {Szarkowska, Agnieszka and Ragni, Valentina and Szkriba, Sonia and Black, Sharon and Orrego-Carmona, David and Kruger, Jan-Louis},
  title = {Watching subtitled videos with the sound off affects viewers’ comprehension, cognitive load, immersion, enjoyment, and gaze patterns: A mixed-methods eye-tracking study},
  journal = {PLOS ONE},
  year = {2024},
  volume = {19},
  number = {10},
  pages = {e0306251},
  doi = {10.1371/journal.pone.0306251}
}
Kostenlos starten →