EUVIMEDEUVIMED entdecken
Open Access

Understanding increments in model performance metrics

Michael J Pencina, Ralph B D'Agostino, Joseph M Massaro

Lifetime Data Analysis · 2013-04 · 19 (2) · 202-218

DOI: 10.1007/s10985-012-9238-0 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Lifetime Data Analysis, online erschienen am 16.12.2012. Verfasst von Michael J Pencina und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics, Harvard Clinical Research Institute, Boston University, CrossTown, 801 Massachusetts Ave., Boston, MA 02118, USA. mpenci…. Hauptthemen laut MEDLINE: Area Under Curve, Models, Statistical. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NHLBI NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The area under the receiver operating characteristic curve (AUC) is the most commonly reported measure of discrimination for prediction models with binary outcomes. However, recently it has been criticized for its inability to increase when important risk factors are added to a baseline model with good discrimination. This has led to the claim that the reliance on the AUC as a measure of discrimination may miss important improvements in clinical performance of risk prediction rules derived from a baseline model. In this paper we investigate this claim by relating the AUC to measures of clinical performance based on sensitivity and specificity under the assumption of multivariate normality. The behavior of the AUC is contrasted with that of discrimination slope. We show that unless rules with very good specificity are desired, the change in the AUC does an adequate job as a predictor of the change in measures of clinical performance. However, stronger or more numerous predictors are needed to achieve the same increment in the AUC for baseline models with good versus poor discrimination. When excellent specificity is desired, our results suggest that the discrimination slope might be a better measure of model improvement than AUC. The theoretical results are illustrated using a Framingham Heart Study example of a model for predicting the 10-year incidence of atrial fibrillation.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Area Under CurveModels, StatisticalAdultAgedAged, 80 and overCardiovascular DiseasesDiscriminant AnalysisFemaleHumansMaleMiddle AgedProspective StudiesRisk Assessment

Autor:innen und Institutionen

  • Department of Biostatistics, Harvard Clinical Research Institute, Boston University, CrossTown, 801 Massachusetts Ave., Boston, MA 02118, USA. mpencina@bu.edu

Förderung

  • NHLBI NIH HHS

Zitieren

APA

Pencina, M. J., D'Agostino, R. B., & Massaro, J. M. (2013). Understanding increments in model performance metrics. Lifetime Data Analysis, 19(2), 202-218. https://doi.org/10.1007/s10985-012-9238-0

Vancouver

Pencina MJ, D'Agostino RB, Massaro JM. Understanding increments in model performance metrics. Lifetime Data Analysis. 2013;19(2):202-218. doi:10.1007/s10985-012-9238-0

BibTeX
@article{Pencina2013,
  author = {Pencina, Michael J and D'Agostino, Ralph B and Massaro, Joseph M},
  title = {Understanding increments in model performance metrics},
  journal = {Lifetime Data Analysis},
  year = {2013},
  volume = {19},
  number = {2},
  pages = {202--218},
  doi = {10.1007/s10985-012-9238-0}
}
Kostenlos starten →