Cheaper and Better: Selecting Good Workers for Crowdsourcing
Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · 23.09.2015 · 3 · 20-21
DOI: 10.1609/hcomp.v3i1.13248 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, online erschienen am 23.09.2015. Verfasst von Hongwei Li und 1 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref).
Abstract (Originalsprache)
Crowdsourcing provides a popular paradigm for data collection at scale. We study the problem of selecting subsets of workers from a given worker pool to maximize the accuracy under a budget constraint. One natural question is whether we should hire as many workers as the budget allows, or restrict on a small number of top-quality workers. By theoretically analyzing the error rate of a typical setting in crowdsourcing, we frame the worker selection problem into a combinatorial optimization problem and propose an algorithm to solve it efficiently. Empirical results on both simulated and real-world datasets show that our algorithm is able to select a small number of high-quality workers, and performs as good as, sometimes even better than, the much larger crowds as the budget allows.
Zitieren
APA
Li, H., & Liu, Q. (2015). Cheaper and Better: Selecting Good Workers for Crowdsourcing. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 3, 20-21. https://doi.org/10.1609/hcomp.v3i1.13248
Vancouver
Li H, Liu Q. Cheaper and Better: Selecting Good Workers for Crowdsourcing. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. 2015;3:20-21. doi:10.1609/hcomp.v3i1.13248
BibTeX
@article{Li2015,
author = {Li, Hongwei and Liu, Qiang},
title = {Cheaper and Better: Selecting Good Workers for Crowdsourcing},
journal = {Proceedings of the AAAI Conference on Human Computation and Crowdsourcing},
year = {2015},
volume = {3},
pages = {20--21},
doi = {10.1609/hcomp.v3i1.13248}
}
EUVIMED