EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Area under the ROC Curve has the most consistent evaluation for binary classification

Jing Li iD

PLOS ONE · 23.12.2024 · 19 (12) · e0316019

DOI: 10.1371/journal.pone.0316019 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 23.12.2024. Verfasst von Jing Li. Beteiligte Einrichtung: Department of Political Science, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America. Hauptthemen laut MEDLINE: ROC Curve, Models, Statistical, Machine Learning. Die Arbeit wurde bisher 125-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how consistent different metrics are at evaluating models across data of different prevalence while the relationships between different variables and the sample size are kept constant. Analyzing 156 data scenarios, 18 model evaluation metrics and five commonly used machine learning models as well as a naive random guess model, I find that evaluation metrics that are less influenced by prevalence offer more consistent evaluation of individual models and more consistent ranking of a set of models. In particular, Area Under the ROC Curve (AUC) which takes all decision thresholds into account when evaluating models has the smallest variance in evaluating individual models and smallest variance in ranking of a set of models. A close threshold analysis using all possible thresholds for all metrics further supports the hypothesis that considering all decision thresholds helps reduce the variance in model evaluation with respect to prevalence change in data. The results have significant implications for model evaluation and model selection in binary classification tasks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ROC CurveModels, StatisticalMachine LearningArea Under CurveSample SizeData Interpretation, StatisticalEvaluation Studies as Topic

Autor:innen und Institutionen

  • Department of Political Science, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America.

ORCID-Profile: Jing Li

Zitieren

APA

Li, J. (2024). Area under the ROC Curve has the most consistent evaluation for binary classification. PLOS ONE, 19(12), e0316019. https://doi.org/10.1371/journal.pone.0316019

Vancouver

Li J. Area under the ROC Curve has the most consistent evaluation for binary classification. PLOS ONE. 2024;19(12):e0316019. doi:10.1371/journal.pone.0316019

BibTeX
@article{Li2024,
  author = {Li, Jing},
  title = {Area under the ROC Curve has the most consistent evaluation for binary classification},
  journal = {PLOS ONE},
  year = {2024},
  volume = {19},
  number = {12},
  pages = {e0316019},
  doi = {10.1371/journal.pone.0316019}
}
Kostenlos starten →