Moderating with the Mob: Evaluating the Efficacy of Real-Time Crowdsourced Fact-Checking
Journal of Online Trust and Safety · 2021
DOI: 10.54501/jots.v1i1.15 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Online Trust and Safety, 2021. Verfasst von William Godel und 6 weiteren Autor:innen. Die Arbeit wurde bisher 46-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0).
Abstract (Originalsprache)
Reducing the spread of false news remains a challenge for social media platforms, as the current strategy of using third-party fact- checkers lacks the capacity to address both the scale and speed of misinformation diffusion. Research on the “wisdom of the crowds” suggests one possible solution: aggregating the evaluations of ordinary users to assess the veracity of information. In this study, we investigate the effectiveness of a scalable model for real-time crowdsourced fact-checking. We select 135 popular news stories and have them evaluated by both ordinary individuals and professional fact-checkers within 72 hours of publication, producing 12,883 individual evaluations. Although we find that machine learning-based models using the crowd perform better at identifying false news than simple aggregation rules, our results suggest that neither approach is able to perform at the level of professional fact-checkers. Additionally, both methods perform best when using evaluations only from survey respondents with high political knowledge, suggesting reason for caution for crowdsourced models that rely on a representative sample of the population. Overall, our analyses reveal that while crowd-based systems provide some information on news quality, they are nonetheless limited—and have significant variation—in their ability to identify false news.
Zitieren
APA
Godel, W., Sanderson, Z., Aslett, K., Nagler, J., Bonneau, R., Persily, N., & Tucker, J. (2021). Moderating with the Mob: Evaluating the Efficacy of Real-Time Crowdsourced Fact-Checking. Journal of Online Trust and Safety. https://doi.org/10.54501/jots.v1i1.15
Vancouver
Godel W, Sanderson Z, Aslett K, Nagler J, Bonneau R, Persily N, et al. Moderating with the Mob: Evaluating the Efficacy of Real-Time Crowdsourced Fact-Checking. Journal of Online Trust and Safety. 2021. doi:10.54501/jots.v1i1.15
BibTeX
@article{Godel2021,
author = {Godel, William and Sanderson, Zeve and Aslett, Kevin and Nagler, Jonathan and Bonneau, Richard and Persily, Nathaniel and Tucker, Joshua},
title = {Moderating with the Mob: Evaluating the Efficacy of Real-Time Crowdsourced Fact-Checking},
journal = {Journal of Online Trust and Safety},
year = {2021},
doi = {10.54501/jots.v1i1.15}
}
EUVIMED