Random forests with parametric entropy-based information gains for classification and regression problems
PeerJ Computer Science · 03.01.2024 · 10 · e1775
DOI: 10.7717/peerj-cs.1775 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 03.01.2024. Verfasst von Vera Ignatenko und 2 weiteren Autor:innen. Beteiligte Einrichtung: Social and Cognitive Informatics Laboratory, National Research University Higher School of Economics, Saint-Petersburg, Russia. Die Arbeit wurde bisher 77-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: The Basic Research Program at the National Research University Higher School of Economics in 2023.
Abstract (Originalsprache)
The random forest algorithm is one of the most popular and commonly used algorithms for classification and regression tasks. It combines the output of multiple decision trees to form a single result. Random forest algorithms demonstrate the highest accuracy on tabular data compared to other algorithms in various applications. However, random forests and, more precisely, decision trees, are usually built with the application of classic Shannon entropy. In this article, we consider the potential of deformed entropies, which are successfully used in the field of complex systems, to increase the prediction accuracy of random forest algorithms. We develop and introduce the information gains based on Renyi, Tsallis, and Sharma-Mittal entropies for classification and regression random forests. We test the proposed algorithm modifications on six benchmark datasets: three for classification and three for regression problems. For classification problems, the application of Renyi entropy allows us to improve the random forest prediction accuracy by 19–96% in dependence on the dataset, Tsallis entropy improves the accuracy by 20–98%, and Sharma-Mittal entropy improves accuracy by 22–111% compared to the classical algorithm. For regression problems, the application of deformed entropies improves the prediction by 2–23% in terms of R2 in dependence on the dataset.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Social and Cognitive Informatics Laboratory, National Research University Higher School of Economics, Saint-Petersburg, Russia.
Förderung
- The Basic Research Program at the National Research University Higher School of Economics in 2023
Zitieren
APA
Ignatenko, V., Surkov, A., & Koltcov, S. (2024). Random forests with parametric entropy-based information gains for classification and regression problems. PeerJ Computer Science, 10, e1775. https://doi.org/10.7717/peerj-cs.1775
Vancouver
Ignatenko V, Surkov A, Koltcov S. Random forests with parametric entropy-based information gains for classification and regression problems. PeerJ Computer Science. 2024;10:e1775. doi:10.7717/peerj-cs.1775
BibTeX
@article{Ignatenko2024,
author = {Ignatenko, Vera and Surkov, Anton and Koltcov, Sergei},
title = {Random forests with parametric entropy-based information gains for classification and regression problems},
journal = {PeerJ Computer Science},
year = {2024},
volume = {10},
pages = {e1775},
doi = {10.7717/peerj-cs.1775}
}
EUVIMED