EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Remote data collection and cognitive task performance in the age of internet bad actors

Kristine D O'Laughlin, Joshua J Volponi, Britte Haugan Cheng, Adam Gazzaley, Joaquin A Anguera

Acta Psychologica · 2026

DOI: 10.1016/j.actpsy.2026.107378 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Acta Psychologica, 2026. Verfasst von Kristine D O'Laughlin und 4 weiteren Autor:innen. Beteiligte Einrichtung: Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States. Hauptthemen laut MEDLINE: Cognition, Data Collection, Executive Function, Internet. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The Chan Zuckerberg Initiative.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Bad actors and bots present threats to data integrity in remote research designs, compromising estimates of cognitive task performance. Here, we share our experiences and recommendations for identifying bad actors and bots and preventing them from enrolling in remote studies of cognitive task performance. As part of a separate large-scale validation study, 3488 participants were recruited through various platforms, including social media and MTurk, and were compensated for their participation. When inspecting these data, we observed a high incidence of unusual or nonsensical responses to demographics questions in combination with unusual cognitive performance patterns. In response, participants were categorized according to bad actor status as determined by 1) number and risk-level associated with red flags within initial screening and pattern of responses; and 2) a researcher review process to identify patterns of aberrant, patterned, or inconsistent responding. Participants were either automatically labeled as bad actors (n = 2829), labeled as bad actors after researcher review (n = 242), or labeled as good actors after researcher review (n = 417). Cognitive task performance was assessed with Adaptive Cognitive Evaluation-Explorer (ACE-X), a mobile, gamified collection of executive function tasks including measures of working memory, attention, and cognitive flexibility. We considered differences between good and bad actors for mean response times, variability in response times, accuracy, and number of trials attempted. Across 6 of the 7 ACE-X tasks evaluated, bad actors responded slower than good actors on average (β = 34.10-150.15, p 0.999), and responded to fewer trials (β = -1.30-0.00, p 0.999). Findings presented here suggest that bad actors show consistent differences in their patterns of performance on cognitive assessments, and therefore careful consideration related to study design and protocols to ensure data integrity are needed when evaluating cognitive performance in remote research settings.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

CognitionData CollectionExecutive FunctionInternetTask Performance and AnalysisHumansFemaleMaleAdultYoung AdultNeuropsychological TestsMemory, Short-TermDigital Media

Stichwörter der Autor:innen

Bad actorsCognitive assessmentData collectionDigitalRemote trial

Autor:innen und Institutionen

  • Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States.
  • Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States; Menlo Education Research, LLC, Menlo Park, CA, United States.
  • Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States; Departments of Neurology, Psychiatry and Physiology, University of Calif…
  • Neuroscape, Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, United States; Department of Psychiatry, University of California, San Francisco, San F…

Förderung

  • The Chan Zuckerberg Initiative

Zitieren

APA

O'Laughlin, K. D., Volponi, J. J., Cheng, B. H., Gazzaley, A., & Anguera, J. A. (2026). Remote data collection and cognitive task performance in the age of internet bad actors. Acta Psychologica. https://doi.org/10.1016/j.actpsy.2026.107378

Vancouver

O'Laughlin KD, Volponi JJ, Cheng BH, Gazzaley A, Anguera JA. Remote data collection and cognitive task performance in the age of internet bad actors. Acta Psychologica. 2026. doi:10.1016/j.actpsy.2026.107378

BibTeX
@article{OLaughlin2026,
  author = {O'Laughlin, Kristine D and Volponi, Joshua J and Cheng, Britte Haugan and Gazzaley, Adam and Anguera, Joaquin A},
  title = {Remote data collection and cognitive task performance in the age of internet bad actors},
  journal = {Acta Psychologica},
  year = {2026},
  doi = {10.1016/j.actpsy.2026.107378}
}
Kostenlos starten →