Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models
Bioinformatics · 01.07.2026 · 42 (Supplement_1)
DOI: 10.1093/bioinformatics/btag226 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 07.07.2026. Verfasst von Ella Rannon und 1 weiteren Autor:innen. Beteiligte Einrichtung: The Shmunis School of Biomedicine and Cancer research, George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, 6997801, Israel. Hauptthemen laut MEDLINE: Proteins, Amino Acids, Computational Biology. Die Arbeit nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Israel Science Foundation, Tel Aviv University Center for AI and Data Science, Edmond J. Safra Center for Bioinformatics at Tel Aviv University.
Abstract (Originalsprache)
Motivation Protein language models (pLMs) typically tokenize sequences at the single-amino-acid level using a 20-residue alphabet, resulting in long input sequences and high computational cost. Sub-word tokenization methods such as Byte Pair Encoding (BPE) can reduce sequence length but are limited by the sparsity of long patterns in proteins encoded by the standard amino acid alphabet. Reduced amino acid alphabets, which group residues by physicochemical properties, offer a potential solution but their performances with sub-word tokenization have not been systematically studied. Results We investigate the combined use of reduced amino acid alphabets and BPE tokenization in protein language models. We pre-train RoBERTa-based pLMs de novo using multiple reduced alphabets and evaluate them across diverse downstream tasks. Our results show that reduced alphabets enable substantially shorter input sequences and faster training and inference. These findings suggest that alphabet reduction may facilitate more effective sub-word tokenization, enabling increased efficiency with marginal impact on predictive performance, and for specific tasks even improving accuracy. Availability and implementation Models, tokenizers, and code are available at github.com/burstein-lab/BioTokenizers.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Substanzen
Autor:innen und Institutionen
- The Shmunis School of Biomedicine and Cancer research, George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, 6997801, Israel.
ORCID-Profile: David Burstein
Förderung
- Israel Science Foundation
- Tel Aviv University Center for AI and Data Science
- Edmond J. Safra Center for Bioinformatics at Tel Aviv University
Zitieren
APA
Rannon, E., & Burstein, D. (2026). Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models. Bioinformatics, 42(Supplement_1). https://doi.org/10.1093/bioinformatics/btag226
Vancouver
Rannon E, Burstein D. Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models. Bioinformatics. 2026;42(Supplement_1). doi:10.1093/bioinformatics/btag226
BibTeX
@article{Rannon2026,
author = {Rannon, Ella and Burstein, David},
title = {Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models},
journal = {Bioinformatics},
year = {2026},
volume = {42},
number = {Supplement_1},
doi = {10.1093/bioinformatics/btag226}
}
EUVIMED