Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation
Procedia Computer Science · 2026 · 283 · 1686-1699
DOI: 10.1016/j.procs.2026.06.244 ↗Kurz gefasst
Fachartikel in der Zeitschrift Procedia Computer Science, 2026. Verfasst von Sravan Kumar Reddy SiddiReddy und 5 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Kein Abstract verfügbar.
Zitieren
APA
SiddiReddy, S. K. R., Sheshadri, S. K., Gupta, D., Paul, B., Bhavani, J. S., & Nair, P. C. (2026). Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation. Procedia Computer Science, 283, 1686-1699. https://doi.org/10.1016/j.procs.2026.06.244
Vancouver
SiddiReddy SKR, Sheshadri SK, Gupta D, Paul B, Bhavani JS, Nair PC. Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation. Procedia Computer Science. 2026;283:1686-1699. doi:10.1016/j.procs.2026.06.244
BibTeX
@article{SiddiReddy2026,
author = {SiddiReddy, Sravan Kumar Reddy and Sheshadri, Shailashree K and Gupta, Deepa and Paul, Biswajit and Bhavani, J Siva and Nair, Priyanka C},
title = {Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation},
journal = {Procedia Computer Science},
year = {2026},
volume = {283},
pages = {1686--1699},
doi = {10.1016/j.procs.2026.06.244}
}
EUVIMED