Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA
PLOS ONE · 14.03.2023 · 18 (3) · e0279720
DOI: 10.1371/journal.pone.0279720 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift PLOS ONE, online erschienen am 14.03.2023. Verfasst von Benjamin D Douglas und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, University of Wisconsin-Madison, Madison, Wisconsin, United States of America. Hauptthemen laut MEDLINE: Data Accuracy, Research Subjects. Die Arbeit wurde bisher 1549-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: BRITE Lab Grant: Behavioral Research Insights Through Experiments Lab,.
Abstract (Originalsprache)
With the proliferation of online data collection in human-subjects research, concerns have been raised over the presence of inattentive survey participants and non-human respondents (bots). We compared the quality of the data collected through five commonly used platforms. Data quality was indicated by the percentage of participants who meaningfully respond to the researcher’s question (high quality) versus those who only contribute noise (low quality). We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items. We divided the samples into high- and low-quality respondents and computed the cost we paid per high-quality respondent. Prolific ($1.90) and CloudResearch ($2.00) were cheaper than MTurk ($4.36) and Qualtrics ($8.17). SONA cost $0.00, yet took the longest to collect the data.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Psychology, University of Wisconsin-Madison, Madison, Wisconsin, United States of America.
- Department of Psychology, Kenyon College, Gambier, Ohio, United States of America.
ORCID-Profile: Markus Brauer
Förderung
- BRITE Lab Grant: Behavioral Research Insights Through Experiments Lab,
Zitieren
APA
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023). Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. PLOS ONE, 18(3), e0279720. https://doi.org/10.1371/journal.pone.0279720
Vancouver
Douglas BD, Ewell PJ, Brauer M. Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. PLOS ONE. 2023;18(3):e0279720. doi:10.1371/journal.pone.0279720
BibTeX
@article{Douglas2023,
author = {Douglas, Benjamin D and Ewell, Patrick J and Brauer, Markus},
title = {Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA},
journal = {PLOS ONE},
year = {2023},
volume = {18},
number = {3},
pages = {e0279720},
doi = {10.1371/journal.pone.0279720}
}
EUVIMED