EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification

Remus Sibisanu, Monica Leba, Andreea Ionica

Entropy · 19.11.2026 · 28 (11) · 1275

DOI: 10.3390/e28111275 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Entropy, online erschienen am 19.11.2026. Verfasst von Remus Sibisanu und 2 weiteren Autor:innen. Die Arbeit nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Classification of rest versus mental arithmetic in the public EEGMAT dataset is often reported with accuracies of 88–99%. Such results usually come from subject-dependent designs, in which epochs from the same person appear in both training and test sets. This design is often intentional and can be appropriate for within-subject prediction. It becomes misleading, however, when the resulting accuracy is read as evidence that a model will work on unseen subjects. We tested how much of the reported accuracy survives a stricter evaluation. Balancing and augmentation were confined to the training data, test sets kept their natural class ratio, and models were selected on validation data. A multilayer perceptron on 57 simple time-domain features was evaluated under four protocols, with over 1000 trained networks and subject-level confidence intervals. Balancing before splitting gave kappa 0.77–0.88, with 45–47% of test items duplicated in training. A correct random epoch split gave kappa 0.29–0.58, so duplicate contamination explains most of the inflation. Leave-one-subject-out evaluation gave kappa 0.19–0.26 and ROC-AUC 0.66–0.71, which is above chance but modest. The remaining subject-dependent advantage (kappa difference 0.06–0.32) is consistent with subject-identity leakage but does not prove it. To our knowledge, no previous study on EEGMAT has examined this question. Results refer to the available 36-subject pool, not to the general population.

Zitieren

APA

Sibisanu, R., Leba, M., & Ionica, A. (2026). High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification. Entropy, 28(11), 1275. https://doi.org/10.3390/e28111275

Vancouver

Sibisanu R, Leba M, Ionica A. High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification. Entropy. 2026;28(11):1275. doi:10.3390/e28111275

BibTeX
@article{Sibisanu2026,
  author = {Sibisanu, Remus and Leba, Monica and Ionica, Andreea},
  title = {High Accuracy Is Not Generalization: Evaluation Design in EEG Mental Arithmetic Classification},
  journal = {Entropy},
  year = {2026},
  volume = {28},
  number = {11},
  pages = {1275},
  doi = {10.3390/e28111275}
}
Kostenlos starten →