EUVIMEDEUVIMED entdecken
Crossref

One-Step Limits of Randomized Speculative Decoding

Hossein Javidnia

ACM AI Letters · 05.11.2026

DOI: 10.1145/3858916 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM AI Letters, online erschienen am 05.11.2026. Verfasst von Hossein Javidnia. Die Arbeit nennt 31 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Speculative decoding accelerates language-model inference by proposing candidate continuations and verifying them in parallel. This letter studies a deliberately restricted one-step abstraction: under a parallel verification budget \(P\) , a non-anticipating policy proposes up to \(P\) continuations, and the accepted length is the longest verifier-consistent prefix. We show that, for any verifier prefix law and any deterministic or randomized policy, the probability of accepting at least \(\ell\) tokens is bounded by the top- \(P\) mass \(M_{P}(\ell)\) of the verifier's length- \(\ell\) prefix distribution. Consequently, \(\mathbb{E}[L_{P}]\leq\sum_{\ell\geq 1}M_{P}(\ell)\) . The result identifies covered verifier-prefix mass as the depthwise bottleneck in this one-step model: randomization cannot improve on the best deterministic prefix cover at any fixed prefix length. Under independent-token specializations, Rényi and collision-entropy bounds yield logarithmic-in- \(P\) limits, while uniform and spread-out regimes give matching or near-matching deterministic constructions. Stylized simulations and inference-only logits experiments illustrate the predicted diminishing returns.

Zitieren

APA

Javidnia, H. (2026). One-Step Limits of Randomized Speculative Decoding. ACM AI Letters. https://doi.org/10.1145/3858916

Vancouver

Javidnia H. One-Step Limits of Randomized Speculative Decoding. ACM AI Letters. 2026. doi:10.1145/3858916

BibTeX
@article{Javidnia2026,
  author = {Javidnia, Hossein},
  title = {One-Step Limits of Randomized Speculative Decoding},
  journal = {ACM AI Letters},
  year = {2026},
  doi = {10.1145/3858916}
}
Kostenlos starten →