Interpretable linear dimensionality reduction based on bias-variance analysis
Data Mining and Knowledge Discovery · 2024-07 · 38 (4) · 1713-1781
DOI: 10.1007/s10618-024-01015-0 ↗Kurz gefasst
Fachartikel in der Zeitschrift Data Mining and Knowledge Discovery, online erschienen am 25.03.2024. Verfasst von Paolo Bonetti und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: H2020 Societal Challenges, Politecnico di Milano.
Abstract (Originalsprache)
One of the central issues of several machine learning applications on real data is the choice of the input features. Ideally, the designer should select a small number of the relevant, nonredundant features to preserve the complete information contained in the original dataset, with little collinearity among features. This procedure helps mitigate problems like overfitting and the curse of dimensionality, which arise when dealing with high-dimensional problems. On the other hand, it is not desirable to simply discard some features, since they may still contain information that can be exploited to improve results. Instead, dimensionality reduction techniques are designed to limit the number of features in a dataset by projecting them into a lower dimensional space, possibly considering all the original features. However, the projected features resulting from the application of dimensionality reduction techniques are usually difficult to interpret. In this paper, we seek to design a principled dimensionality reduction approach that maintains the interpretability of the resulting features. Specifically, we propose a bias-variance analysis for linear models and we leverage these theoretical results to design an algorithm, Linear Correlated Features Aggregation (LinCFA), which aggregates groups of continuous features with their average if their correlation is “sufficiently large”. In this way, all features are considered, the dimensionality is reduced and the interpretability is preserved. Finally, we provide numerical validations of the proposed algorithm both on synthetic datasets to confirm the theoretical results and on real datasets to show some promising applications.
Förderung
- H2020 Societal Challenges
- Politecnico di Milano
Zitieren
APA
Bonetti, P., Metelli, A. M., & Restelli, M. (2024). Interpretable linear dimensionality reduction based on bias-variance analysis. Data Mining and Knowledge Discovery, 38(4), 1713-1781. https://doi.org/10.1007/s10618-024-01015-0
Vancouver
Bonetti P, Metelli AM, Restelli M. Interpretable linear dimensionality reduction based on bias-variance analysis. Data Mining and Knowledge Discovery. 2024;38(4):1713-1781. doi:10.1007/s10618-024-01015-0
BibTeX
@article{Bonetti2024,
author = {Bonetti, Paolo and Metelli, Alberto Maria and Restelli, Marcello},
title = {Interpretable linear dimensionality reduction based on bias-variance analysis},
journal = {Data Mining and Knowledge Discovery},
year = {2024},
volume = {38},
number = {4},
pages = {1713--1781},
doi = {10.1007/s10618-024-01015-0}
}
EUVIMED