Evaluating Estimator Performance Under Multicollinearity: A Trade-Off Between MSE and Accuracy in Logistic, Lasso, Elastic Net, and Ridge Regression with Varying Penalty Parameters
Stats · 31.05.2025 · 8 (2) · 45
DOI: 10.3390/stats8020045 ↗Kurz gefasst
Fachartikel in der Zeitschrift Stats, online erschienen am 31.05.2025. Verfasst von H. M. Nayem und 2 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 59 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Multicollinearity in logistic regression models can result in inflated variances and yield unreliable estimates of parameters. Ridge regression, a regularized estimation technique, is frequently employed to address this issue. This study conducts a comparative evaluation of the performance of 23 established ridge regression estimators alongside Logistic Regression, Elastic-Net, Lasso, and Generalized Ridge Regression (GRR), considering various levels of multicollinearity within the context of logistic regression settings. Simulated datasets with high correlations (0.80, 0.90, 0.95, and 0.99) and real-world data (municipal and cancer remission) were analyzed. Both results show that ridge estimators, such as kAL1, kAL2, kKL1, and kKL2, exhibit strong performance in terms of Mean Squared Error (MSE) and accuracy, particularly in smaller samples, while GRR demonstrates superior performance in large samples. Real-world data further confirm that GRR achieves the lowest MSE in highly collinear municipal data, while ridge estimators and GRR help prevent overfitting in small-sample cancer remission data. The results underscore the efficacy of ridge estimators and GRR in handling multicollinearity, offering reliable alternatives to traditional regression techniques, especially for datasets with high correlations and varying sample sizes.
Zitieren
APA
Nayem, H. M., Aziz, S., & Kibria, B. M. G. (2025). Evaluating Estimator Performance Under Multicollinearity: A Trade-Off Between MSE and Accuracy in Logistic, Lasso, Elastic Net, and Ridge Regression with Varying Penalty Parameters. Stats, 8(2), 45. https://doi.org/10.3390/stats8020045
Vancouver
Nayem HM, Aziz S, Kibria BMG. Evaluating Estimator Performance Under Multicollinearity: A Trade-Off Between MSE and Accuracy in Logistic, Lasso, Elastic Net, and Ridge Regression with Varying Penalty Parameters. Stats. 2025;8(2):45. doi:10.3390/stats8020045
BibTeX
@article{Nayem2025,
author = {Nayem, H. M. and Aziz, Sinha and Kibria, B. M. Golam},
title = {Evaluating Estimator Performance Under Multicollinearity: A Trade-Off Between MSE and Accuracy in Logistic, Lasso, Elastic Net, and Ridge Regression with Varying Penalty Parameters},
journal = {Stats},
year = {2025},
volume = {8},
number = {2},
pages = {45},
doi = {10.3390/stats8020045}
}
EUVIMED