Evolving LLMs from Next-Token Prediction to Multi-Token Prediction via Self-Distillation
Electronics · 2026-04-06 · 15 (7) · 1533
DOI: 10.3390/electronics15071533 ↗At a glance
Journal article in Electronics, published online on 2026-04-06. Written by Yang Xu and 1 co-authors. 44 references. The full text is freely available (open access, CC BY 4.0). Funded by: National Natural Science Foundation of China.
Abstract (original language)
Mainstream Large Language Models (LLMs) work under the paradigm of Next-Token Prediction (NTP). Multi-Token Prediction (MTP) is motivated by higher decoding efficiency, extending NTP to enable LLMs to draft multiple tokens during each forward pass. However, existing MTP approaches pretrain MTP along with the target LLM, making it difficult to unlock MTP for LLMs without official support. In this work, we propose a post-hoc approach to training an MTP module for a target LLM, providing an efficient way to evolve the LLM from NTP to MTP. The proposed approach features two main characteristics. (1) No changes to the target LLM, since it is frozen during MTP training. (2) Efficient MTP training via self-distillation from the target LLM’s native NTP capability. Results show that our approach can post-hoc train a performant MTP module via lightweight pretraining.
Funding
- National Natural Science Foundation of China
Cite
APA
Xu, Y., & Che, W. (2026). Evolving LLMs from Next-Token Prediction to Multi-Token Prediction via Self-Distillation. Electronics, 15(7), 1533. https://doi.org/10.3390/electronics15071533
Vancouver
Xu Y, Che W. Evolving LLMs from Next-Token Prediction to Multi-Token Prediction via Self-Distillation. Electronics. 2026;15(7):1533. doi:10.3390/electronics15071533
BibTeX
@article{Xu2026,
author = {Xu, Yang and Che, Wanxiang},
title = {Evolving LLMs from Next-Token Prediction to Multi-Token Prediction via Self-Distillation},
journal = {Electronics},
year = {2026},
volume = {15},
number = {7},
pages = {1533},
doi = {10.3390/electronics15071533}
}
EUVIMED