A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning
Journal of Educational and Behavioral Statistics · 2024-04 · 49 (2) · 151-172
DOI: 10.3102/10769986231171711 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Educational and Behavioral Statistics, online erschienen am 10.05.2023. Verfasst von Youmi Suk und 1 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 59 Literaturangaben. Gefördert von: National Science Foundation.
Abstract (Originalsprache)
As algorithmic decision making is increasingly deployed in every walk of life, many researchers have raised concerns about fairness-related bias from such algorithms. But there is little research on harnessing psychometric methods to uncover potential discriminatory bias inside decision-making algorithms. The main goal of this article is to propose a new framework for algorithmic fairness based on differential item functioning (DIF), which has been commonly used to measure item fairness in psychometrics. Our fairness notion, which we call differential algorithmic functioning (DAF), is defined based on three pieces of information: a decision variable, a “fair” variable, and a protected variable such as race or gender. Under the DAF framework, an algorithm can exhibit uniform DAF, nonuniform DAF, or neither (i.e., non-DAF). For detecting DAF, we provide modifications of well-established DIF methods: Mantel–Haenszel test, logistic regression, and residual-based DIF. We demonstrate our framework through a real dataset concerning decision-making algorithms for grade retention in K–12 education in the United States.
Förderung
- National Science Foundation
Zitieren
APA
Suk, Y., & Han, K. T. (2024). A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning. Journal of Educational and Behavioral Statistics, 49(2), 151-172. https://doi.org/10.3102/10769986231171711
Vancouver
Suk Y, Han KT. A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning. Journal of Educational and Behavioral Statistics. 2024;49(2):151-172. doi:10.3102/10769986231171711
BibTeX
@article{Suk2024,
author = {Suk, Youmi and Han, Kyung T.},
title = {A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning},
journal = {Journal of Educational and Behavioral Statistics},
year = {2024},
volume = {49},
number = {2},
pages = {151--172},
doi = {10.3102/10769986231171711}
}
EUVIMED