EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Assessing the efficacy of a participant-vetting procedure to improve data-quality on Amazon’s Mechanical Turk

Emilio D. Rivera, Benjamin M. Wilkowski, Aaron J. Moss, Cheskie Rosenzweig, Leib Litman

Methodology · 30.06.2022 · 18 (2) · 126-143

DOI: 10.5964/meth.8331 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Methodology, online erschienen am 30.06.2022. Verfasst von Emilio D. Rivera und 4 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, Amazon’s Mechanical Turk (MTurk) has become a pivotal source for participant recruitment in many social-science fields. In the last several years, however, concerns about data quality have arisen. In response, CloudResearch developed an intensive pre-screening procedure to vet the full participant pool available on MTurk and exclude those providing low-quality data. To assess its efficacy, we compared three MTurk samples that completed identical measures: Sample 1 was collected prior to the pre-screening’s implementation. Sample 2 was collected shortly following its implementation, and Sample 3 was collected nearly a full-year after its implementation. Results indicated that the reliability and validity of scales improved with the implementation of this prescreening procedure, and that this was especially apparent with more recent versions. Thus, this prescreening procedure appears to be a valuable tool to help ensure the collection of high-quality data on MTurk.

Zitieren

APA

Rivera, E. D., Wilkowski, B. M., Moss, A. J., Rosenzweig, C., & Litman, L. (2022). Assessing the efficacy of a participant-vetting procedure to improve data-quality on Amazon’s Mechanical Turk. Methodology, 18(2), 126-143. https://doi.org/10.5964/meth.8331

Vancouver

Rivera ED, Wilkowski BM, Moss AJ, Rosenzweig C, Litman L. Assessing the efficacy of a participant-vetting procedure to improve data-quality on Amazon’s Mechanical Turk. Methodology. 2022;18(2):126-143. doi:10.5964/meth.8331

BibTeX
@article{Rivera2022,
  author = {Rivera, Emilio D. and Wilkowski, Benjamin M. and Moss, Aaron J. and Rosenzweig, Cheskie and Litman, Leib},
  title = {Assessing the efficacy of a participant-vetting procedure to improve data-quality on Amazon’s Mechanical Turk},
  journal = {Methodology},
  year = {2022},
  volume = {18},
  number = {2},
  pages = {126--143},
  doi = {10.5964/meth.8331}
}
Kostenlos starten →