Toward a Perspectivist Turn in Ground Truthing for Predictive Computing
Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (6) · 6860-6868
DOI: 10.1609/aaai.v37i6.25840 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Federico Cabitza und 2 weiteren Autor:innen. Die Arbeit wurde bisher 57-mal zitiert (Crossref).
Abstract (Originalsprache)
Most current Artificial Intelligence applications are based on supervised Machine Learning (ML), which ultimately grounds on data annotated by small teams of experts or large ensemble of volunteers. The annotation process is often performed in terms of a majority vote, however this has been proved to be often problematic by recent evaluation studies. In this article, we describe and advocate for a different paradigm, which we call perspectivism: this counters the removal of disagreement and, consequently, the assumption of correctness of traditionally aggregated gold-standard datasets, and proposes the adoption of methods that preserve divergence of opinions and integrate multiple perspectives in the ground truthing process of ML development. Drawing on previous works which inspired it, mainly from the crowdsourcing and multi-rater labeling settings, we survey the state-of-the-art and describe the potential of our proposal for not only the more subjective tasks (e.g. those related to human language) but also those tasks commonly understood as objective (e.g. medical decision making). We present the main benefits of adopting a perspectivist stance in ML, as well as possible disadvantages, and various ways in which such a stance can be implemented in practice. Finally, we share a set of recommendations and outline a research agenda to advance the perspectivist stance in ML.
Zitieren
APA
Cabitza, F., Campagner, A., & Basile, V. (2023). Toward a Perspectivist Turn in Ground Truthing for Predictive Computing. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6), 6860-6868. https://doi.org/10.1609/aaai.v37i6.25840
Vancouver
Cabitza F, Campagner A, Basile V. Toward a Perspectivist Turn in Ground Truthing for Predictive Computing. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(6):6860-6868. doi:10.1609/aaai.v37i6.25840
BibTeX
@article{Cabitza2023,
author = {Cabitza, Federico and Campagner, Andrea and Basile, Valerio},
title = {Toward a Perspectivist Turn in Ground Truthing for Predictive Computing},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2023},
volume = {37},
number = {6},
pages = {6860--6868},
doi = {10.1609/aaai.v37i6.25840}
}
EUVIMED