EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Separation-like irregularity and sample size optimism in high-discrimination logistic prediction models

Ye Liang, Louis Shuo Wang iD, Jiguang Yu, Xin Zan

PLOS One · 24.08.2026 · 21 (8) · e0342286

DOI: 10.1371/journal.pone.0342286 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 24.08.2026. Verfasst von Ye Liang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Industrial and Systems Engineering, University of Iowa, Iowa City, Iowa, United States of America. Hauptthemen laut MEDLINE: Prediction Algorithms. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Closed-form minimum sample size criteria for developing logistic prediction models, such as the Riley framework implemented in pmsampsize , are widely used but may become optimistic when anticipated discrimination is high. We conducted a Monte Carlo simulation study to compare the formula-based recommended development sample size, n R i l e y , with an empirical required sample size, n r e q , defined by out-of-sample calibration-slope stability under repeated development sampling. Scenarios fixed the candidate parameter dimension at p = 10 and crossed predictor distribution (normal, standardized skewed continuous, binary), signal density (dense versus sparse), prevalence ( ϕ ∈ { 0.05 , 0.10 , 0.20 } ), and target discrimination ( A U C t a r g e t ∈ { 0.70 , 0.75 , 0.80 , 0.85 , 0.90 } ), with intercept and signal strength calibrated to match targets. We defined n r e q as the smallest n such that 𝔼 ( b n ) ≥ 0.90 and Pr ( b n < 0.80 ) ≤ 0.20 , where b n is the truth-based logit-scale calibration slope evaluated on a large fixed validation covariate set of size N v a l = 50 , 000 . At moderate discrimination, n R i l e y approximated n r e q , but as discrimination increased the formula increasingly underestimated the sample size required for calibration stability, with large deficits at A U C t a r g e t = 0.90 . Separation-like behavior (extreme fitted risks and linear predictors) at n = n R i l e y became common in high-discrimination settings despite nominal convergence, providing a plausible mechanism for formula optimism. These findings support augmenting formula-based planning with targeted simulation stress tests and instability diagnostics when high discrimination is anticipated.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Prediction AlgorithmsHumansComputer SimulationLogistic ModelsMonte Carlo MethodSample Size

Autor:innen und Institutionen

  • Department of Industrial and Systems Engineering, University of Iowa, Iowa City, Iowa, United States of America.
  • Department of Mathematics, University of Tennessee, Knoxville, Tennessee, United States of America.
  • College of Engineering, Boston University, Boston, Massachusetts, United States of America.

ORCID-Profile: Louis Shuo Wang

Zitieren

APA

Liang, Y., Wang, L. S., Yu, J., & Zan, X. (2026). Separation-like irregularity and sample size optimism in high-discrimination logistic prediction models. PLOS One, 21(8), e0342286. https://doi.org/10.1371/journal.pone.0342286

Vancouver

Liang Y, Wang LS, Yu J, Zan X. Separation-like irregularity and sample size optimism in high-discrimination logistic prediction models. PLOS One. 2026;21(8):e0342286. doi:10.1371/journal.pone.0342286

BibTeX
@article{Liang2026,
  author = {Liang, Ye and Wang, Louis Shuo and Yu, Jiguang and Zan, Xin},
  title = {Separation-like irregularity and sample size optimism in high-discrimination logistic prediction models},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {8},
  pages = {e0342286},
  doi = {10.1371/journal.pone.0342286}
}
Kostenlos starten →