A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning
Sensors · 20.02.2023 · 23 (4) · 2333
DOI: 10.3390/s23042333 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 20.02.2023. Verfasst von Szilvia Szeghalmy und 1 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Informatics, University of Debrecen, H-4028 Debrecen, Hungary. Die Arbeit wurde bisher 172-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Nowadays, the solution to many practical problems relies on machine learning tools. However, compiling the appropriate training data set for real-world classification problems is challenging because collecting the right amount of data for each class is often difficult or even impossible. In such cases, we can easily face the problem of imbalanced learning. There are many methods in the literature for solving the imbalanced learning problem, so it has become a serious question how to compare the performance of the imbalanced learning methods. Inadequate validation techniques can provide misleading results (e.g., due to data shift), which leads to the development of methods designed for imbalanced data sets, such as stratified cross-validation (SCV) and distribution optimally balanced SCV (DOB-SCV). Previous studies have shown that higher classification performance scores (AUC) can be achieved on imbalanced data sets using DOB-SCV instead of SCV. We investigated the effect of the oversamplers on this difference. The study was conducted on 420 data sets, involving several sampling methods and the DTree, kNN, SVM, and MLP classifiers. We point out that DOB-SCV often provides a little higher F1 and AUC values for classification combined with sampling. However, the results also prove that the selection of the sampler–classifier pair is more important for the classification performance than the choice between the DOB-SCV and the SCV techniques.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Informatics, University of Debrecen, H-4028 Debrecen, Hungary.
ORCID-Profile: Szilvia Szeghalmy, Attila Fazekas
Zitieren
APA
Szeghalmy, S., & Fazekas, A. (2023). A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning. Sensors, 23(4), 2333. https://doi.org/10.3390/s23042333
Vancouver
Szeghalmy S, Fazekas A. A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning. Sensors. 2023;23(4):2333. doi:10.3390/s23042333
BibTeX
@article{Szeghalmy2023,
author = {Szeghalmy, Szilvia and Fazekas, Attila},
title = {A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning},
journal = {Sensors},
year = {2023},
volume = {23},
number = {4},
pages = {2333},
doi = {10.3390/s23042333}
}
EUVIMED