EUVIMEDEUVIMED entdecken
Crossref

Interactive Consensus Agreement Games for Labeling Images

Paul Upchurch, Daniel Sedra, Andrew Mullen, Haym Hirsh, Kavita Bala

Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · 21.09.2016 · 4 · 239-248

DOI: 10.1609/hcomp.v4i1.13293 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, online erschienen am 21.09.2016. Verfasst von Paul Upchurch und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Scene understanding algorithms in computer vision are improving dramatically by training deep convolutional neural networks on millions of accurately annotated images. Collecting large-scale datasets for this kind of training is challenging, and the learning algorithms are only as good as the data they train on. Training annotations are often obtained by taking the majority label from independent crowdsourced workers using platforms such as Amazon Mechanical Turk. However, the accuracy of the resulting annotations can vary, with the hardest-to-annotate samples having prohibitively low accuracy. Our insight is that in cases where independent worker annotations are poor more accurate results can be obtained by having workers collaborate. This paper introduces consensus agreement games, a novel method for assigning annotations to images by the agreement of multiple consensuses of small cliques of workers. We demonstrate that this approach reduces error by 37.8% on two different datasets at a cost of $0.10 or $0.17 per annotation. The higher cost is justified because our method does not need to be run on the entire dataset. Ultimately, our method enables us to more accurately annotate images and build more challenging training datasets for learning algorithms.

Zitieren

APA

Upchurch, P., Sedra, D., Mullen, A., Hirsh, H., & Bala, K. (2016). Interactive Consensus Agreement Games for Labeling Images. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 4, 239-248. https://doi.org/10.1609/hcomp.v4i1.13293

Vancouver

Upchurch P, Sedra D, Mullen A, Hirsh H, Bala K. Interactive Consensus Agreement Games for Labeling Images. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. 2016;4:239-248. doi:10.1609/hcomp.v4i1.13293

BibTeX
@article{Upchurch2016,
  author = {Upchurch, Paul and Sedra, Daniel and Mullen, Andrew and Hirsh, Haym and Bala, Kavita},
  title = {Interactive Consensus Agreement Games for Labeling Images},
  journal = {Proceedings of the AAAI Conference on Human Computation and Crowdsourcing},
  year = {2016},
  volume = {4},
  pages = {239--248},
  doi = {10.1609/hcomp.v4i1.13293}
}
Kostenlos starten →