EUVIMEDSkopri EUVIMED
ValidierungsstudieOpen access · CC BY-NC-ND 4.0

Validation of a Risk‐Prediction Model in the Presence of Outcome Misclassification

Runjia Zou, Brian D Williamson iD, Susan M Shortreed, R Yates Coley iD

Statistics in Medicine · 2026-04 · 45 (8-9)

DOI: 10.1002/sim.70377 ↗

At a glance

Validierungsstudie in Statistics in Medicine, published online on 2026-04-03. Written by Runjia Zou and 3 co-authors. Institution: Department of Biostatistics, University of Washington, Seattle, Washington, USA. Major MEDLINE topics: Models, Statistical, Electronic Health Records. Cited 1 times so far (Crossref); 48 references. The full text is freely available (open access, CC BY-NC-ND 4.0). Funded by: NIMH NIH HHS, National Institute of Mental Health.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Electronic health records (EHRs) provide a rich data source for building prediction models to improve the quality of care. However, EHR data are prone to measurement error, and outcomes used to evaluate prediction model performance may be misclassified. Comparing risk predictions to misclassified outcomes will result in unreliable estimates of a prediction model's performance. We propose a method that leverages a smaller chart review sample with gold‐standard outcome measurements to adjust validation for outcome misclassification and provide more accurate prediction model evaluation. We derive formulae to estimate true positive rate (TPR), false positive rate (FPR), positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC) in the presence of outcome misclassification. Different scenarios of misclassification are explored, including when the misclassification is independent or dependent on features, and when misclassification is unidirectional (e.g., only missed diagnoses) or bidirectional. In simulation studies, we compare the bias and 95% confidence interval coverage of performance estimates obtained using our proposed method to those estimated with misclassified outcomes (without accounting for misclassification) or in the smaller chart review sample. Simulation results indicate that, across all misclassification scenarios examined, our proposed estimates have good accuracy and improved precision. Outcome misclassification should be considered when evaluating a prediction model's performance in order to accurately inform decision‐making about whether and how to use a clinical prediction model.

Subjects

MeSH terms (MEDLINE)

Models, StatisticalElectronic Health RecordsHumansPrediction AlgorithmsComputer SimulationROC CurveArea Under CurveBiasRisk AssessmentPredictive Value of TestsPredictive Learning Models

Author keywords

ROC curveclassification accuracyclinical prediction modelmachine learningmeasurement error

Authors and institutions

  • Department of Biostatistics, University of Washington, Seattle, Washington, USA.
  • Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, Washington, USA.
  • Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Center, Seattle, Washington, USA.
  • Department of Health Systems and Population Health, University of Washington, Seattle, Washington, USA.

ORCID profiles: Brian D Williamson, R Yates Coley

Funding

  • NIMH NIH HHS
  • National Institute of Mental Health

Cite

APA

Zou, R., D Williamson, B., M Shortreed, S., & Coley, R. Y. (2026). Validation of a Risk‐Prediction Model in the Presence of Outcome Misclassification. Statistics in Medicine, 45(8-9). https://doi.org/10.1002/sim.70377

Vancouver

Zou R, D Williamson B, M Shortreed S, Coley RY. Validation of a Risk‐Prediction Model in the Presence of Outcome Misclassification. Statistics in Medicine. 2026;45(8-9). doi:10.1002/sim.70377

BibTeX
@article{Zou2026,
  author = {Zou, Runjia and D Williamson, Brian and M Shortreed, Susan and Coley, R Yates},
  title = {Validation of a Risk‐Prediction Model in the Presence of Outcome Misclassification},
  journal = {Statistics in Medicine},
  year = {2026},
  volume = {45},
  number = {8-9},
  doi = {10.1002/sim.70377}
}
Start for free →