EUVIMEDEUVIMED entdecken
Open Access

A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study

Haoyuan Wang, Chuan Hong, Michael Pencina, Matthew Engelhard

JMIR AI · 24.08.2026 · 5 · e85822-e85822

DOI: 10.2196/85822 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR AI, online erschienen am 24.08.2026. Verfasst von Haoyuan Wang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics and Bioinformatics, Duke University School of Medicine, 2424 Erwin Rd, Durham, NC, 27705, United States, 1 919-613-3665. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIMH NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Fairness evaluation is essential for trustworthy clinical risk prediction. However, existing fairness-oriented discrimination metrics either ignore cross-group comparisons or rely on exhaustive pairwise evaluations, making them difficult to interpret and impractical for model selection.

Objective

This study aimed to develop and evaluate novel fairness-oriented discrimination metrics for clinical risk prediction that address limitations of within-group and pairwise cross-group approaches.

Methods

We examined theoretical properties of existing U-statistic–based metrics, including concordance index (CI) and area under the receiver operating characteristic curve (AUC), when applied to subgroups. We highlighted the distinction between within-group discrimination (ranking within a subgroup) and group-level discrimination (ranking relative to the broader population). Building on this framework, we proposed group-level extensions of the CI and AUC that summarize subgroup-specific performance in a single interpretable measure. We then applied these metrics to the PREVENT (Predicting Risk of Cardiovascular Disease Events) equation, a recently developed model for atherosclerotic cardiovascular disease.

Results

The traditional subgroup-specific CI and AUC captured within-group but not group-level discrimination, obscuring inequities in clinical decision-making. Existing cross-group approaches (eg, the xCI and xAUC metrics) addressed this limitation but became computationally and interpretively burdensome with multiple subgroups due to pairwise comparisons. Our proposed metrics provided a streamlined alternative, yielding 1 summary statistic per subgroup while retaining sensitivity to cross-group ranking disparities. Applied to PREVENT, these metrics revealed differences in subgroup performance not apparent from within-group evaluations.

Conclusions

By distinguishing between within-group and group-level discrimination, our framework clarifies a common source of misinterpretation in fairness evaluation. The proposed group-level extensions of the CI and AUC provide practical, interpretable tools for evaluating fairness in clinical prediction models, enabling more transparent and equitable risk assessment.

Schlagwörter

Stichwörter der Autor:innen

algorithmscardiovascular risk predictionmachine learningperformance metricstrustworthy AI

Autor:innen und Institutionen

  • Department of Biostatistics and Bioinformatics, Duke University School of Medicine, 2424 Erwin Rd, Durham, NC, 27705, United States, 1 919-613-3665.

Förderung

  • NIMH NIH HHS

Zitieren

APA

Wang, H., Hong, C., Pencina, M., & Engelhard, M. (2026). A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study. JMIR AI, 5, e85822-e85822. https://doi.org/10.2196/85822

Vancouver

Wang H, Hong C, Pencina M, Engelhard M. A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study. JMIR AI. 2026;5:e85822-e85822. doi:10.2196/85822

BibTeX
@article{Wang2026,
  author = {Wang, Haoyuan and Hong, Chuan and Pencina, Michael and Engelhard, Matthew},
  title = {A Simplified Metric to Streamline Between-Group Fairness Assessment for Predictive Models: Algorithm Development and Evaluation Study},
  journal = {JMIR AI},
  year = {2026},
  volume = {5},
  pages = {e85822--e85822},
  doi = {10.2196/85822}
}
Kostenlos starten →