EUVIMEDEUVIMED entdecken
Crossref

Grokking Larger DLP with Fixed Model Size: O (1) Representations for Scalable Discrete Logarithm Learning

David Vesterlund

International Journal of Mathematic Exploration and Computer Education · 03.11.2026

DOI: 10.47485/3069-9703.1030 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Mathematic Exploration and Computer Education, online erschienen am 03.11.2026. Verfasst von David Vesterlund. Die Arbeit nennt 22 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We study whether the delayed generalization phenomenon (“grokking”) on the finite-field discrete logarithm problem (DLP) can scale to large primes without increasing model size. Our companion paper [Vesterlund, 2026a] demonstrated DLP grokking on small primes (p ≤ 521) but identified a fundamental “tokenizer confound”: both the input embedding and output classifier grow linearly with p, so model parameter count scales as O(p). Here we replace the atomic tokenizer with two O(1) components—Factorized Residue Embedding (FRE) for inputs and Factorized Candidate Output (FCO) for outputs—combined with a CRT-BOTH representation that decomposes the DLP into two coprime subproblems via the Chinese Remainder Theorem. The resulting architecture maintains a fixed ⁓526k parameters regardless of prime size. We demonstrate that this fixed-size model generalizes on DLP across a range of prime sizes (p = 113 to p = 4001), achieving 100% test accuracy at every scale. However, not all configurations exhibit grokking (a memorization plateau followed by delayed generalization): for p = 601 and p = 4001, memorization and generalization times coincide (Tmem = Tgrok), indicating direct generalization rather than grokking. The effective problem size scales with max(a,b) (the larger CRT factor), not with p: the 35× span in p corresponds to a 16× to 256× span in max(a,b). The parameter efficiency advantage grows with p: at p = 4,001 the fixed model is 2.7× smaller than the atomic baseline. Caveat: The CRT decomposition makes per-factor sub-instances few enough that the standard triple-level split allows every sub-instance to appear in both train and test; a sub-instance holdout experiment is needed to determine whether the observed generalization is genuine algorithm learning or sub-instance lookup.

Zitieren

APA

Vesterlund, D. (2026). Grokking Larger DLP with Fixed Model Size: O (1) Representations for Scalable Discrete Logarithm Learning. International Journal of Mathematic Exploration and Computer Education. https://doi.org/10.47485/3069-9703.1030

Vancouver

Vesterlund D. Grokking Larger DLP with Fixed Model Size: O (1) Representations for Scalable Discrete Logarithm Learning. International Journal of Mathematic Exploration and Computer Education. 2026. doi:10.47485/3069-9703.1030

BibTeX
@article{Vesterlund2026,
  author = {Vesterlund, David},
  title = {Grokking Larger DLP with Fixed Model Size: O (1) Representations for Scalable Discrete Logarithm Learning},
  journal = {International Journal of Mathematic Exploration and Computer Education},
  year = {2026},
  doi = {10.47485/3069-9703.1030}
}
Kostenlos starten →