EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Out of (the) bag—encoding categorical predictors impacts out-of-bag samples

Helen L Smith, Patrick J Biggs iD, Nigel P French, Adam N H Smith iD, Jonathan C Marshall iD

PeerJ Computer Science · 18.11.2024 · 10 · e2445

DOI: 10.7717/peerj-cs.2445 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 18.11.2024. Verfasst von Helen L Smith und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Mathematical and Computational Sciences, Massey University, Palmerston North, New Zealand. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Massey University School of Fundamental Sciences scholarship, New Zealand Food Safety Science & Research Centre.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Performance of random forest classification models is often assessed and interpreted using out-of-bag (OOB) samples. Observations which are OOB when a tree is trained may serve as a test set for that tree and predictions from the OOB observations used to calculate OOB error and variable importance measures (VIM). OOB errors are popular because they are fast to compute and, for large samples, are a good estimate of the true prediction error. In this study, we investigate how target-based vs . target-agnostic encoding of categorical predictor variables for random forest can bias performance measures based on OOB samples. We show that, when categorical variables are encoded using a target-based encoding method, and when the encoding takes place prior to bagging, the OOB sample can underestimate the true misclassification rate, and overestimate variable importance. We recommend using a separate test data set when evaluating variable importance and/or predictive performance of tree based methods that utilise a target-based encoding method.

Schlagwörter

Stichwörter der Autor:innen

Absent levelsCategorical predictorsLabel encodingOut-of-bag errorRandom forestVariable importance

Autor:innen und Institutionen

  • School of Mathematical and Computational Sciences, Massey University, Palmerston North, New Zealand.
  • School of Food Technology and Natural Sciences, Massey University, Palmerston North, New Zealand.
  • NZ Food Safety and Science Research Centre, Massey University, Palmerston North, New Zealand.
  • School of Veterinary Science, Massey University, Palmerston North, New Zealand.
  • School of Mathematical and Computational Sciences, Massey University, Auckland, New Zealand.

ORCID-Profile: Patrick J Biggs, Adam N H Smith, Jonathan C Marshall

Förderung

  • Massey University School of Fundamental Sciences scholarship
  • New Zealand Food Safety Science & Research Centre

Zitieren

APA

Smith, H. L., Biggs, P. J., French, N. P., Smith, A. N. H., & Marshall, J. C. (2024). Out of (the) bag—encoding categorical predictors impacts out-of-bag samples. PeerJ Computer Science, 10, e2445. https://doi.org/10.7717/peerj-cs.2445

Vancouver

Smith HL, Biggs PJ, French NP, Smith ANH, Marshall JC. Out of (the) bag—encoding categorical predictors impacts out-of-bag samples. PeerJ Computer Science. 2024;10:e2445. doi:10.7717/peerj-cs.2445

BibTeX
@article{Smith2024,
  author = {Smith, Helen L and Biggs, Patrick J and French, Nigel P and Smith, Adam N H and Marshall, Jonathan C},
  title = {Out of (the) bag—encoding categorical predictors impacts out-of-bag samples},
  journal = {PeerJ Computer Science},
  year = {2024},
  volume = {10},
  pages = {e2445},
  doi = {10.7717/peerj-cs.2445}
}
Kostenlos starten →