An Equal-Byte Non-Inferiority Protocol for Storage-Aware, Reproducible Evaluation of Replay-Based Continual Learning
Machine Learning and Knowledge Extraction · 2026-07-19 · 8 (7) · 212
DOI: 10.3390/make8070212 ↗At a glance
Studienprotokoll in Machine Learning and Knowledge Extraction, published online on 2026-07-19. Written by Lava Kumar Polu and 2 co-authors. 32 references. The full text is freely available (open access, CC BY 4.0).
Abstract (original language)
Replay-based continual learners are commonly evaluated at a fixed item count, but deployment hardware meters memory in bytes, and methods storing logits, feature vectors, or compressed codes occupy different byte footprints at an equal item count. The equal-item convention therefore does not answer the deployment question. We propose an equal-byte non-inferiority protocol that holds total stored bytes constant across competitors, selects the strongest of a pre-registered eight-method baseline pool at each budget, and reports a decision at a registered margin of ΔNI=0.05 under Holm–Bonferroni correction. We demonstrate it on Split-CIFAR-100 and on an eight-task family-grouped sequence of fifty curated Abstraction and Reasoning Corpus puzzles, using a two-tier consolidation memory with a byte-budget knob, at ten seeds per cell. Under the pre-registered rule, the two-tier scheme fails non-inferiority at all four budgets on both benchmarks, the strongest per-buffer baseline exceeds the best two-tier cell at every budget, and the outcome is robust to task ordering on the abstract-reasoning benchmark. An interpretability analysis links the outcome to the compressed tier’s attention share and prototype similarity. The contribution is the protocol; the demonstration returns a clean negative decision on one specific candidate.
Cite
APA
Polu, L. K., Vaida, M. L., & Syal, A. (2026). An Equal-Byte Non-Inferiority Protocol for Storage-Aware, Reproducible Evaluation of Replay-Based Continual Learning. Machine Learning and Knowledge Extraction, 8(7), 212. https://doi.org/10.3390/make8070212
Vancouver
Polu LK, Vaida ML, Syal A. An Equal-Byte Non-Inferiority Protocol for Storage-Aware, Reproducible Evaluation of Replay-Based Continual Learning. Machine Learning and Knowledge Extraction. 2026;8(7):212. doi:10.3390/make8070212
BibTeX
@article{Polu2026,
author = {Polu, Lava Kumar and Vaida, Maria L. and Syal, Aditya},
title = {An Equal-Byte Non-Inferiority Protocol for Storage-Aware, Reproducible Evaluation of Replay-Based Continual Learning},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {7},
pages = {212},
doi = {10.3390/make8070212}
}
EUVIMED