EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA

Benjamin D Douglas, Patrick J Ewell, Markus Brauer iD

PLOS ONE · 14.03.2023 · 18 (3) · e0279720

DOI: 10.1371/journal.pone.0279720 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift PLOS ONE, online erschienen am 14.03.2023. Verfasst von Benjamin D Douglas und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, University of Wisconsin-Madison, Madison, Wisconsin, United States of America. Hauptthemen laut MEDLINE: Data Accuracy, Research Subjects. Die Arbeit wurde bisher 1549-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: BRITE Lab Grant: Behavioral Research Insights Through Experiments Lab,.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With the proliferation of online data collection in human-subjects research, concerns have been raised over the presence of inattentive survey participants and non-human respondents (bots). We compared the quality of the data collected through five commonly used platforms. Data quality was indicated by the percentage of participants who meaningfully respond to the researcher’s question (high quality) versus those who only contribute noise (low quality). We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items. We divided the samples into high- and low-quality respondents and computed the cost we paid per high-quality respondent. Prolific ($1.90) and CloudResearch ($2.00) were cheaper than MTurk ($4.36) and Qualtrics ($8.17). SONA cost $0.00, yet took the longest to collect the data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Data AccuracyResearch SubjectsHumansSurveys and Questionnaires

Autor:innen und Institutionen

  • Department of Psychology, University of Wisconsin-Madison, Madison, Wisconsin, United States of America.
  • Department of Psychology, Kenyon College, Gambier, Ohio, United States of America.

ORCID-Profile: Markus Brauer

Förderung

  • BRITE Lab Grant: Behavioral Research Insights Through Experiments Lab,

Zitieren

APA

Douglas, B. D., Ewell, P. J., & Brauer, M. (2023). Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. PLOS ONE, 18(3), e0279720. https://doi.org/10.1371/journal.pone.0279720

Vancouver

Douglas BD, Ewell PJ, Brauer M. Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. PLOS ONE. 2023;18(3):e0279720. doi:10.1371/journal.pone.0279720

BibTeX
@article{Douglas2023,
  author = {Douglas, Benjamin D and Ewell, Patrick J and Brauer, Markus},
  title = {Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA},
  journal = {PLOS ONE},
  year = {2023},
  volume = {18},
  number = {3},
  pages = {e0279720},
  doi = {10.1371/journal.pone.0279720}
}
Kostenlos starten →