Evaluating Model Performance Under Worst-Case Subpopulations
Mathematics of Operations Research · 19.08.2026
DOI: 10.1287/moor.2024.0803 ↗Kurz gefasst
Fachartikel in der Zeitschrift Mathematics of Operations Research, online erschienen am 19.08.2026. Verfasst von Mike Li und 3 weiteren Autor:innen.
Abstract (Originalsprache)
The performance of machine learning models degrades when the training population is different from that seen under operation. Toward assessing distributional robustness, we study the worst-case performance of a model over all subpopulations of a given size defined with respect to core attributes Z. This notion of robustness can consider arbitrary (continuous) attributes Z and automatically accounts for complex intersectionality in disadvantaged groups. We develop a scalable yet principled two-stage estimation procedure that can evaluate the robustness of state-of-the-art models. We prove that our procedure enjoys several finite-sample convergence guarantees, including dimension-free convergence. Instead of overly conservative notions based on Rademacher complexities, our evaluation error depends on the dimension of Z only through the out-of-sample error in estimating the performance conditional on Z. On real data sets, we demonstrate that our method certifies the robustness of a model and prevents deployment of unreliable models.
Zitieren
APA
Li, M., Mittal, D., Namkoong, H., & Xia, S. (2026). Evaluating Model Performance Under Worst-Case Subpopulations. Mathematics of Operations Research. https://doi.org/10.1287/moor.2024.0803
Vancouver
Li M, Mittal D, Namkoong H, Xia S. Evaluating Model Performance Under Worst-Case Subpopulations. Mathematics of Operations Research. 2026. doi:10.1287/moor.2024.0803
BibTeX
@article{Li2026,
author = {Li, Mike and Mittal, Daksh and Namkoong, Hongseok and Xia, Shangzhou},
title = {Evaluating Model Performance Under Worst-Case Subpopulations},
journal = {Mathematics of Operations Research},
year = {2026},
doi = {10.1287/moor.2024.0803}
}
EUVIMED