EUVIMEDEUVIMED entdecken
Randomisierte kontrollierte StudieOpen Access

Bot or Not? Detecting and Managing Participant Deception When Conducting Digital Research Remotely: Case Study of a Randomized Controlled Trial

Gemma Loebenberg iD, Melissa Oldham iD, Jamie Brown iD, Larisa Dinu iD, Susan Michie iD, Matt Field iD, Felix Greaves iD, Claire Garnett iD

Journal of Medical Internet Research · 2023

DOI: 10.2196/46523 ↗

Kurz gefasst

Randomisierte kontrollierte Studie in der Zeitschrift Journal of Medical Internet Research, 2023. Verfasst von Gemma Loebenberg und 7 weiteren Autor:innen. Beteiligte Einrichtung: UCL Tobacco and Alcohol Research Group, University College London, London, United Kingdom. Hauptthemen laut MEDLINE: Software, Cell Phone. Die Arbeit wurde bisher 50-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Department of Health.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Evaluating digital interventions using remote methods enables the recruitment of large numbers of participants relatively conveniently and cheaply compared with in-person methods. However, conducting research remotely based on participant self-report with little verification is open to automated “bots” and participant deception.

Objective

This paper uses a case study of a remotely conducted trial of an alcohol reduction app to highlight and discuss (1) the issues with participant deception affecting remote research trials with financial compensation; and (2) the importance of rigorous data management to detect and address these issues.

Methods

We recruited participants on the internet from July 2020 to March 2022 for a randomized controlled trial (n=5602) evaluating the effectiveness of an alcohol reduction app, Drink Less. Follow-up occurred at 3 time points, with financial compensation offered (up to £36 [US $39.23]). Address authentication and telephone verification were used to detect 2 kinds of deception: “bots,” that is, automated responses generated in clusters; and manual participant deception, that is, participants providing false information.

Results

Of the 1142 participants who enrolled in the first 2 months of recruitment, 75.6% (n=863) of them were identified as bots during data screening. As a result, a CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart) was added, and after this, no more bots were identified. Manual participant deception occurred throughout the study. Of the 5956 participants (excluding bots) who enrolled in the study, 298 (5%) were identified as false participants. The extent of this decreased from 110 in November 2020, to a negligible level by February 2022 including a number of months with 0. The decline occurred after we added further screening questions such as attention checks, removed the prominence of financial compensation from social media advertising, and added an additional requirement to provide a mobile phone number for identity verification.

Conclusions

Data management protocols are necessary to detect automated bots and manual participant deception in remotely conducted trials. Bots and manual deception can be minimized by adding a CAPTCHA, attention checks, a requirement to provide a phone number for identity verification, and not prominently advertising financial compensation on social media. Trial Registration ISRCTN Number ISRCTN64052601; https://doi.org/10.1186/ISRCTN64052601

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SoftwareCell PhoneHumansAdvertisingData ManagementEthanolDeception

Stichwörter der Autor:innen

artificial intelligencefalse informationmHealth applicationsparticipantparticipant deceptionrecruitresearch subjectweb-based studies

Substanzen

Ethanol

Autor:innen und Institutionen

  • UCL Tobacco and Alcohol Research Group, University College London, London, United Kingdom.
  • Clinical Educational and Health Psychology, University College London, London, United Kingdom.
  • Department of Psychology, University of Sheffield, Sheffield, United Kingdom.
  • Department of Primary Care and Public Health, Imperial College London, London, United Kingdom.

ORCID-Profile: Gemma Loebenberg, Melissa Oldham, Jamie Brown, Larisa Dinu, Susan Michie, Matt Field, Felix Greaves, Claire Garnett

Förderung

  • Department of Health

Zitieren

APA

Loebenberg, G., Oldham, M., Brown, J., Dinu, L., Michie, S., Field, M., Greaves, F., & Garnett, C. (2023). Bot or Not? Detecting and Managing Participant Deception When Conducting Digital Research Remotely: Case Study of a Randomized Controlled Trial. Journal of Medical Internet Research. https://doi.org/10.2196/46523

Vancouver

Loebenberg G, Oldham M, Brown J, Dinu L, Michie S, Field M, et al. Bot or Not? Detecting and Managing Participant Deception When Conducting Digital Research Remotely: Case Study of a Randomized Controlled Trial. Journal of Medical Internet Research. 2023. doi:10.2196/46523

BibTeX
@article{Loebenberg2023,
  author = {Loebenberg, Gemma and Oldham, Melissa and Brown, Jamie and Dinu, Larisa and Michie, Susan and Field, Matt and Greaves, Felix and Garnett, Claire},
  title = {Bot or Not? Detecting and Managing Participant Deception When Conducting Digital Research Remotely: Case Study of a Randomized Controlled Trial},
  journal = {Journal of Medical Internet Research},
  year = {2023},
  doi = {10.2196/46523}
}
Kostenlos starten →