The formal rationality of artificial intelligence-based algorithms and the problem of bias
Journal of Information Technology · 2024-03 · 39 (1) · 19-40
DOI: 10.1177/02683962231176842 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Information Technology, online erschienen am 29.05.2023. Verfasst von Rohit Nishant und 2 weiteren Autor:innen. Die Arbeit wurde bisher 113-mal zitiert (Crossref) und nennt 74 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
This paper presents a new perspective on the problem of bias in artificial intelligence (AI)-driven decision-making by examining the fundamental difference between AI and human rationality in making sense of data. Current research has focused primarily on software engineers’ bounded rationality and bias in the data fed to algorithms but has neglected the crucial role of algorithmic rationality in producing bias. Using a Weberian distinction between formal and substantive rationality, we inquire why AI-based algorithms lack the ability to display common sense in data interpretation, leading to flawed decisions. We first conduct a rigorous text analysis to uncover and exemplify contextual nuances within the sampled data. We then combine unsupervised and supervised learning, revealing that algorithmic decision-making characterizes and judges data categories mechanically as it operates through the formal rationality of mathematical optimization procedures. Next, using an AI tool, we demonstrate how formal rationality embedded in AI-based algorithms limits its capacity to perform adequately in complex contexts, thus leading to bias and poor decisions. Finally, we delineate the boundary conditions and limitations of leveraging formal rationality to automatize algorithmic decision-making. Our study provides a deeper understanding of the rationality-based causes of AI’s role in bias and poor decisions, even when data is generated in a largely bias-free context.
Zitieren
APA
Nishant, R., Schneckenberg, D., & Ravishankar, M. (2024). The formal rationality of artificial intelligence-based algorithms and the problem of bias. Journal of Information Technology, 39(1), 19-40. https://doi.org/10.1177/02683962231176842
Vancouver
Nishant R, Schneckenberg D, Ravishankar M. The formal rationality of artificial intelligence-based algorithms and the problem of bias. Journal of Information Technology. 2024;39(1):19-40. doi:10.1177/02683962231176842
BibTeX
@article{Nishant2024,
author = {Nishant, Rohit and Schneckenberg, Dirk and Ravishankar, MN},
title = {The formal rationality of artificial intelligence-based algorithms and the problem of bias},
journal = {Journal of Information Technology},
year = {2024},
volume = {39},
number = {1},
pages = {19--40},
doi = {10.1177/02683962231176842}
}
EUVIMED