EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Interpretable linear dimensionality reduction based on bias-variance analysis

Paolo Bonetti, Alberto Maria Metelli, Marcello Restelli

Data Mining and Knowledge Discovery · 2024-07 · 38 (4) · 1713-1781

DOI: 10.1007/s10618-024-01015-0 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Data Mining and Knowledge Discovery, online erschienen am 25.03.2024. Verfasst von Paolo Bonetti und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: H2020 Societal Challenges, Politecnico di Milano.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

One of the central issues of several machine learning applications on real data is the choice of the input features. Ideally, the designer should select a small number of the relevant, nonredundant features to preserve the complete information contained in the original dataset, with little collinearity among features. This procedure helps mitigate problems like overfitting and the curse of dimensionality, which arise when dealing with high-dimensional problems. On the other hand, it is not desirable to simply discard some features, since they may still contain information that can be exploited to improve results. Instead, dimensionality reduction techniques are designed to limit the number of features in a dataset by projecting them into a lower dimensional space, possibly considering all the original features. However, the projected features resulting from the application of dimensionality reduction techniques are usually difficult to interpret. In this paper, we seek to design a principled dimensionality reduction approach that maintains the interpretability of the resulting features. Specifically, we propose a bias-variance analysis for linear models and we leverage these theoretical results to design an algorithm, Linear Correlated Features Aggregation (LinCFA), which aggregates groups of continuous features with their average if their correlation is “sufficiently large”. In this way, all features are considered, the dimensionality is reduced and the interpretability is preserved. Finally, we provide numerical validations of the proposed algorithm both on synthetic datasets to confirm the theoretical results and on real datasets to show some promising applications.

Förderung

  • H2020 Societal Challenges
  • Politecnico di Milano

Zitieren

APA

Bonetti, P., Metelli, A. M., & Restelli, M. (2024). Interpretable linear dimensionality reduction based on bias-variance analysis. Data Mining and Knowledge Discovery, 38(4), 1713-1781. https://doi.org/10.1007/s10618-024-01015-0

Vancouver

Bonetti P, Metelli AM, Restelli M. Interpretable linear dimensionality reduction based on bias-variance analysis. Data Mining and Knowledge Discovery. 2024;38(4):1713-1781. doi:10.1007/s10618-024-01015-0

BibTeX
@article{Bonetti2024,
  author = {Bonetti, Paolo and Metelli, Alberto Maria and Restelli, Marcello},
  title = {Interpretable linear dimensionality reduction based on bias-variance analysis},
  journal = {Data Mining and Knowledge Discovery},
  year = {2024},
  volume = {38},
  number = {4},
  pages = {1713--1781},
  doi = {10.1007/s10618-024-01015-0}
}
Kostenlos starten →