EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Random forests with parametric entropy-based information gains for classification and regression problems

Vera Ignatenko, Anton Surkov, Sergei Koltcov

PeerJ Computer Science · 03.01.2024 · 10 · e1775

DOI: 10.7717/peerj-cs.1775 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 03.01.2024. Verfasst von Vera Ignatenko und 2 weiteren Autor:innen. Beteiligte Einrichtung: Social and Cognitive Informatics Laboratory, National Research University Higher School of Economics, Saint-Petersburg, Russia. Die Arbeit wurde bisher 77-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The Basic Research Program at the National Research University Higher School of Economics in 2023.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The random forest algorithm is one of the most popular and commonly used algorithms for classification and regression tasks. It combines the output of multiple decision trees to form a single result. Random forest algorithms demonstrate the highest accuracy on tabular data compared to other algorithms in various applications. However, random forests and, more precisely, decision trees, are usually built with the application of classic Shannon entropy. In this article, we consider the potential of deformed entropies, which are successfully used in the field of complex systems, to increase the prediction accuracy of random forest algorithms. We develop and introduce the information gains based on Renyi, Tsallis, and Sharma-Mittal entropies for classification and regression random forests. We test the proposed algorithm modifications on six benchmark datasets: three for classification and three for regression problems. For classification problems, the application of Renyi entropy allows us to improve the random forest prediction accuracy by 19–96% in dependence on the dataset, Tsallis entropy improves the accuracy by 20–98%, and Sharma-Mittal entropy improves accuracy by 22–111% compared to the classical algorithm. For regression problems, the application of deformed entropies improves the prediction by 2–23% in terms of R2 in dependence on the dataset.

Schlagwörter

Stichwörter der Autor:innen

ClassificationRandom forestRegressionRenyi entropySharma-Mittal entropyTsallis entropy

Autor:innen und Institutionen

  • Social and Cognitive Informatics Laboratory, National Research University Higher School of Economics, Saint-Petersburg, Russia.

Förderung

  • The Basic Research Program at the National Research University Higher School of Economics in 2023

Zitieren

APA

Ignatenko, V., Surkov, A., & Koltcov, S. (2024). Random forests with parametric entropy-based information gains for classification and regression problems. PeerJ Computer Science, 10, e1775. https://doi.org/10.7717/peerj-cs.1775

Vancouver

Ignatenko V, Surkov A, Koltcov S. Random forests with parametric entropy-based information gains for classification and regression problems. PeerJ Computer Science. 2024;10:e1775. doi:10.7717/peerj-cs.1775

BibTeX
@article{Ignatenko2024,
  author = {Ignatenko, Vera and Surkov, Anton and Koltcov, Sergei},
  title = {Random forests with parametric entropy-based information gains for classification and regression problems},
  journal = {PeerJ Computer Science},
  year = {2024},
  volume = {10},
  pages = {e1775},
  doi = {10.7717/peerj-cs.1775}
}
Kostenlos starten →