High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification
Entropy · 19.11.2026 · 28 (11) · 1275
DOI: 10.3390/e28111275 ↗Kurz gefasst
Fachartikel in der Zeitschrift Entropy, online erschienen am 19.11.2026. Verfasst von Remus Sibisanu und 2 weiteren Autor:innen. Die Arbeit nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Classification of rest versus mental arithmetic in the public EEGMAT dataset is often reported with accuracies of 88–99%. Such results usually come from subject-dependent designs, in which epochs from the same person appear in both training and test sets. This design is often intentional and can be appropriate for within-subject prediction. It becomes misleading, however, when the resulting accuracy is read as evidence that a model will work on unseen subjects. We tested how much of the reported accuracy survives a stricter evaluation. Balancing and augmentation were confined to the training data, test sets kept their natural class ratio, and models were selected on validation data. A multilayer perceptron on 57 simple time-domain features was evaluated under four protocols, with over 1000 trained networks and subject-level confidence intervals. Balancing before splitting gave kappa 0.77–0.88, with 45–47% of test items duplicated in training. A correct random epoch split gave kappa 0.29–0.58, so duplicate contamination explains most of the inflation. Leave-one-subject-out evaluation gave kappa 0.19–0.26 and ROC-AUC 0.66–0.71, which is above chance but modest. The remaining subject-dependent advantage (kappa difference 0.06–0.32) is consistent with subject-identity leakage but does not prove it. To our knowledge, no previous study on EEGMAT has examined this question. Results refer to the available 36-subject pool, not to the general population.
Zitieren
APA
Sibisanu, R., Leba, M., & Ionica, A. (2026). High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification. Entropy, 28(11), 1275. https://doi.org/10.3390/e28111275
Vancouver
Sibisanu R, Leba M, Ionica A. High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification. Entropy. 2026;28(11):1275. doi:10.3390/e28111275
BibTeX
@article{Sibisanu2026,
author = {Sibisanu, Remus and Leba, Monica and Ionica, Andreea},
title = {High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification},
journal = {Entropy},
year = {2026},
volume = {28},
number = {11},
pages = {1275},
doi = {10.3390/e28111275}
}
EUVIMED