A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study
JMIR AI · 24.08.2026 · 5 · e85822-e85822
DOI: 10.2196/85822 ↗Kurz gefasst
Fachartikel in der Zeitschrift JMIR AI, online erschienen am 24.08.2026. Verfasst von Haoyuan Wang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics and Bioinformatics, Duke University School of Medicine, 2424 Erwin Rd, Durham, NC, 27705, United States, 1 919-613-3665. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIMH NIH HHS.
Abstract (Originalsprache)
Background
Fairness evaluation is essential for trustworthy clinical risk prediction. However, existing fairness-oriented discrimination metrics either ignore cross-group comparisons or rely on exhaustive pairwise evaluations, making them difficult to interpret and impractical for model selection.
Objective
This study aimed to develop and evaluate novel fairness-oriented discrimination metrics for clinical risk prediction that address limitations of within-group and pairwise cross-group approaches.
Methods
We examined theoretical properties of existing U-statistic–based metrics, including concordance index (CI) and area under the receiver operating characteristic curve (AUC), when applied to subgroups. We highlighted the distinction between within-group discrimination (ranking within a subgroup) and group-level discrimination (ranking relative to the broader population). Building on this framework, we proposed group-level extensions of the CI and AUC that summarize subgroup-specific performance in a single interpretable measure. We then applied these metrics to the PREVENT (Predicting Risk of Cardiovascular Disease Events) equation, a recently developed model for atherosclerotic cardiovascular disease.
Results
The traditional subgroup-specific CI and AUC captured within-group but not group-level discrimination, obscuring inequities in clinical decision-making. Existing cross-group approaches (eg, the xCI and xAUC metrics) addressed this limitation but became computationally and interpretively burdensome with multiple subgroups due to pairwise comparisons. Our proposed metrics provided a streamlined alternative, yielding 1 summary statistic per subgroup while retaining sensitivity to cross-group ranking disparities. Applied to PREVENT, these metrics revealed differences in subgroup performance not apparent from within-group evaluations.
Conclusions
By distinguishing between within-group and group-level discrimination, our framework clarifies a common source of misinterpretation in fairness evaluation. The proposed group-level extensions of the CI and AUC provide practical, interpretable tools for evaluating fairness in clinical prediction models, enabling more transparent and equitable risk assessment.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, 2424 Erwin Rd, Durham, NC, 27705, United States, 1 919-613-3665.
Förderung
- NIMH NIH HHS
Zitieren
APA
Wang, H., Hong, C., Pencina, M., & Engelhard, M. (2026). A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study. JMIR AI, 5, e85822-e85822. https://doi.org/10.2196/85822
Vancouver
Wang H, Hong C, Pencina M, Engelhard M. A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study. JMIR AI. 2026;5:e85822-e85822. doi:10.2196/85822
BibTeX
@article{Wang2026,
author = {Wang, Haoyuan and Hong, Chuan and Pencina, Michael and Engelhard, Matthew},
title = {A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study},
journal = {JMIR AI},
year = {2026},
volume = {5},
pages = {e85822--e85822},
doi = {10.2196/85822}
}
EUVIMED