Bringing practical statistical science to AI and predictive model fairness testing
AI and Ethics · 2025-06 · 5 (3) · 2149-2164
DOI: 10.1007/s43681-024-00518-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift AI and Ethics, online erschienen am 08.08.2024. Verfasst von Victor S. Y. Lo und 2 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Artificial Intelligence, Machine Learning, Statistical Modeling and Predictive Analytics have been widely used in various industries for a long time. More recently, AI Model Governance including AI Ethics has received significant attention from academia, industry, and regulatory agencies. To minimize potential unjustified treatment disfavoring individuals based on demographics, an increasingly critical task is to assess group fairness through some established metrics. Many commercial and open-source tools are now available to support the computations of these fairness metrics. However, this area is largely based on rules, e.g., metrics within a prespecified range would be considered satisfactory. These metrics are statistical estimates and are often based on limited sample data and therefore subject to sampling variability. For instance, if a fairness criterion is barely met or missed, it is often uncertain if it should be a “pass” or “failure,” if the sample size is not large. This is where statistical science can help. Specifically, statistical hypothesis testing enables us to determine whether the sample data can support a particular hypothesis (e.g., falling within an acceptable range) or the observations may have happened by chance. Drawing upon the bioequivalence literature from medicine and advanced hypothesis testing in statistics, we propose a practical statistical significance testing method to enhance the current rule-based process for model fairness testing and its associated power calculation, followed by an illustration with a realistic example.
Zitieren
APA
Lo, V. S. Y., Datta, S., & Salami, Y. (2025). Bringing practical statistical science to AI and predictive model fairness testing. AI and Ethics, 5(3), 2149-2164. https://doi.org/10.1007/s43681-024-00518-2
Vancouver
Lo VSY, Datta S, Salami Y. Bringing practical statistical science to AI and predictive model fairness testing. AI and Ethics. 2025;5(3):2149-2164. doi:10.1007/s43681-024-00518-2
BibTeX
@article{Lo2025,
author = {Lo, Victor S. Y. and Datta, Sayan and Salami, Youssouf},
title = {Bringing practical statistical science to AI and predictive model fairness testing},
journal = {AI and Ethics},
year = {2025},
volume = {5},
number = {3},
pages = {2149--2164},
doi = {10.1007/s43681-024-00518-2}
}
EUVIMED