EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The Role of Mutual Information Estimator Choice in Feature Selection: An Empirical Study on mRMR

Nikolaos Papaioannou, Georgios Myllis, Alkiviadis Tsimpiris, Vasiliki Vrana

Information · 25.08.2025 · 16 (9) · 724

DOI: 10.3390/info16090724 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information, online erschienen am 25.08.2025. Verfasst von Nikolaos Papaioannou und 3 weiteren Autor:innen. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Maximum Relevance Minimum Redundancy (mRMR) is a widely used feature selection method that is applied in a wide range of applications in various fields. mRMR adds to the optimal subset the features that have high relevance to the target variable while having minimum redundancy with each other. Mutual information is a key component of mRMR as it measures the degree of dependence between two variables. However, the real value of mutual information is not known and needs to be estimated. The aim of this study is to examine whether the choice of mutual information estimator affects the performance of mRMR. To this end, three variations of mRMR are compared. The first one uses Parzen window estimation to assess mutual information between continuous variables. The second is based on equidistant partitioning using the cells method, while the third incorporates a bias-corrected version of the same estimator. All methods are tested with and without a regularization term in the mRMR denominator, introduced to improve numerical stability. The evaluation is conducted on synthetic datasets where the target variable is defined as a combination of continuous features, simulating both linear and nonlinear dependencies. To demonstrate the applicability of the proposed methods, we also include a case study in real-world classification tasks. The study carried out showed that the choice of mutual information estimator can affect the performance of mRMR and it must be carefully selected depending on the dataset and the parameters of the examined problem. The application of the corrected mutual information estimator improves the performance of mRMR in the examined setup.

Zitieren

APA

Papaioannou, N., Myllis, G., Tsimpiris, A., & Vrana, V. (2025). The Role of Mutual Information Estimator Choice in Feature Selection: An Empirical Study on mRMR. Information, 16(9), 724. https://doi.org/10.3390/info16090724

Vancouver

Papaioannou N, Myllis G, Tsimpiris A, Vrana V. The Role of Mutual Information Estimator Choice in Feature Selection: An Empirical Study on mRMR. Information. 2025;16(9):724. doi:10.3390/info16090724

BibTeX
@article{Papaioannou2025,
  author = {Papaioannou, Nikolaos and Myllis, Georgios and Tsimpiris, Alkiviadis and Vrana, Vasiliki},
  title = {The Role of Mutual Information Estimator Choice in Feature Selection: An Empirical Study on mRMR},
  journal = {Information},
  year = {2025},
  volume = {16},
  number = {9},
  pages = {724},
  doi = {10.3390/info16090724}
}
Kostenlos starten →