EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation

Sravan Kumar Reddy SiddiReddy, Shailashree K Sheshadri, Deepa Gupta, Biswajit Paul, J Siva Bhavani, Priyanka C Nair

Procedia Computer Science · 2026 · 283 · 1686-1699

DOI: 10.1016/j.procs.2026.06.244 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Procedia Computer Science, 2026. Verfasst von Sravan Kumar Reddy SiddiReddy und 5 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Kein Abstract verfügbar.

Zitieren

APA

SiddiReddy, S. K. R., Sheshadri, S. K., Gupta, D., Paul, B., Bhavani, J. S., & Nair, P. C. (2026). Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation. Procedia Computer Science, 283, 1686-1699. https://doi.org/10.1016/j.procs.2026.06.244

Vancouver

SiddiReddy SKR, Sheshadri SK, Gupta D, Paul B, Bhavani JS, Nair PC. Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation. Procedia Computer Science. 2026;283:1686-1699. doi:10.1016/j.procs.2026.06.244

BibTeX
@article{SiddiReddy2026,
  author = {SiddiReddy, Sravan Kumar Reddy and Sheshadri, Shailashree K and Gupta, Deepa and Paul, Biswajit and Bhavani, J Siva and Nair, Priyanka C},
  title = {Tokenization Matters: The Impact of Byte and BPE Encoder-Only Language Models on Low-Resource Adaptation in Multilingual Neural Machine Translation},
  journal = {Procedia Computer Science},
  year = {2026},
  volume = {283},
  pages = {1686--1699},
  doi = {10.1016/j.procs.2026.06.244}
}
Kostenlos starten →