EUVIMEDEUVIMED entdecken
Open Access

Measuring the Reliability of a Gamified Stroop Task: Quantitative Experiment

Katelyn Wiley iD, Phaedra Berger iD, Maximilian Achim Friehs iD, Regan Lee Mandryk iD

JMIR Serious Games · 10.04.2024 · 12 · e50315

DOI: 10.2196/50315 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Serious Games, online erschienen am 10.04.2024. Verfasst von Katelyn Wiley und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, University of Saskatchewan, Saskatoon, SK, Canada. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Few gamified cognitive tasks are subjected to rigorous examination of psychometric properties, despite their use in experimental and clinical settings. Even small manipulations to cognitive tasks require extensive research to understand their effects.

Objective

This study aims to investigate how game elements can affect the reliability of scores on a Stroop task. We specifically investigated performance consistency within and across sessions.

Methods

We created 2 versions of the Stroop task, with and without game elements, and then tested each task with participants at 2 time points. The gamified task used points and feedback as game elements. In this paper, we report on the reliability of the gamified Stroop task in terms of internal consistency and test-retest reliability, compared with the control task. We used a permutation approach to evaluate internal consistency. For test-retest reliability, we calculated the Pearson correlation and intraclass correlation coefficients between each time point. We also descriptively compared the reliability of scores on a trial-by-trial basis, considering the different trial types.

Results

At the first time point, the Stroop effect was reduced in the game condition, indicating an increase in performance. Participants in the game condition had faster reaction times (P=.005) and lower error rates (P=.04) than those in the basic task condition. Furthermore, the game condition led to higher measures of internal consistency at both time points for reaction times and error rates, which indicates a more consistent response pattern. For reaction time in the basic task condition, at time 1, rSpearman-Brown=0.78, 95% CI 0.64-0.89. At time 2, rSpearman-Brown=0.64, 95% CI 0.40-0.81. For reaction time, in the game condition, at time 1, rSpearman-Brown=0.83, 95% CI 0.71-0.91. At time 2, rSpearman-Brown=0.76, 95% CI 0.60-0.88. Similarly, for error rates in the basic task condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. For error rates in the game condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. Test-retest reliability analysis revealed a distinctive performance pattern depending on the trial type, which may be reflective of motivational differences between task versions. In short, especially in the incongruent trials where cognitive conflict occurs, performance in the game condition reaches peak consistency after 100 trials, whereas performance consistency drops after 50 trials for the basic version and only catches up to the game after 250 trials.

Conclusions

Even subtle gamification can impact task performance albeit not only in terms of a direct difference in performance between conditions. People playing the game reach peak performance sooner, and their performance is more consistent within and across sessions. We advocate for a closer examination of the impact of game elements on performance.

Schlagwörter

Stichwörter der Autor:innen

Stroop taskcognitive assessmentgamificationreliabilityserious games

Autor:innen und Institutionen

  • Department of Computer Science, University of Saskatchewan, Saskatoon, SK, Canada.
  • Faculty of Behavioural, Management and Social Sciences, University of Twente, Enschede, Netherlands.
  • School of Psychology, University College Dublin, Dublin, Ireland.
  • Max-Planck-Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
  • Department of Computer Science, University of Victoria, Victoria, BC, Canada.

ORCID-Profile: Katelyn Wiley, Phaedra Berger, Maximilian Achim Friehs, Regan Lee Mandryk

Zitieren

APA

Wiley, K., Berger, P., Friehs, M. A., & Mandryk, R. L. (2024). Measuring the Reliability of a Gamified Stroop Task: Quantitative Experiment. JMIR Serious Games, 12, e50315. https://doi.org/10.2196/50315

Vancouver

Wiley K, Berger P, Friehs MA, Mandryk RL. Measuring the Reliability of a Gamified Stroop Task: Quantitative Experiment. JMIR Serious Games. 2024;12:e50315. doi:10.2196/50315

BibTeX
@article{Wiley2024,
  author = {Wiley, Katelyn and Berger, Phaedra and Friehs, Maximilian Achim and Mandryk, Regan Lee},
  title = {Measuring the Reliability of a Gamified Stroop Task: Quantitative Experiment},
  journal = {JMIR Serious Games},
  year = {2024},
  volume = {12},
  pages = {e50315},
  doi = {10.2196/50315}
}
Kostenlos starten →