EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models

Ella Rannon, David Burstein iD

Bioinformatics · 01.07.2026 · 42 (Supplement_1)

DOI: 10.1093/bioinformatics/btag226 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 07.07.2026. Verfasst von Ella Rannon und 1 weiteren Autor:innen. Beteiligte Einrichtung: The Shmunis School of Biomedicine and Cancer research, George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, 6997801, Israel. Hauptthemen laut MEDLINE: Proteins, Amino Acids, Computational Biology. Die Arbeit nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Israel Science Foundation, Tel Aviv University Center for AI and Data Science, Edmond J. Safra Center for Bioinformatics at Tel Aviv University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation Protein language models (pLMs) typically tokenize sequences at the single-amino-acid level using a 20-residue alphabet, resulting in long input sequences and high computational cost. Sub-word tokenization methods such as Byte Pair Encoding (BPE) can reduce sequence length but are limited by the sparsity of long patterns in proteins encoded by the standard amino acid alphabet. Reduced amino acid alphabets, which group residues by physicochemical properties, offer a potential solution but their performances with sub-word tokenization have not been systematically studied. Results We investigate the combined use of reduced amino acid alphabets and BPE tokenization in protein language models. We pre-train RoBERTa-based pLMs de novo using multiple reduced alphabets and evaluate them across diverse downstream tasks. Our results show that reduced alphabets enable substantially shorter input sequences and faster training and inference. These findings suggest that alphabet reduction may facilitate more effective sub-word tokenization, enabling increased efficiency with marginal impact on predictive performance, and for specific tasks even improving accuracy. Availability and implementation Models, tokenizers, and code are available at github.com/burstein-lab/BioTokenizers.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ProteinsAmino AcidsComputational BiologyAmino Acid Sequence

Substanzen

ProteinsAmino Acids

Autor:innen und Institutionen

  • The Shmunis School of Biomedicine and Cancer research, George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, 6997801, Israel.

ORCID-Profile: David Burstein

Förderung

  • Israel Science Foundation
  • Tel Aviv University Center for AI and Data Science
  • Edmond J. Safra Center for Bioinformatics at Tel Aviv University

Zitieren

APA

Rannon, E., & Burstein, D. (2026). Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models. Bioinformatics, 42(Supplement_1). https://doi.org/10.1093/bioinformatics/btag226

Vancouver

Rannon E, Burstein D. Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models. Bioinformatics. 2026;42(Supplement_1). doi:10.1093/bioinformatics/btag226

BibTeX
@article{Rannon2026,
  author = {Rannon, Ella and Burstein, David},
  title = {Optimizing protein tokenization: reduced amino acid alphabets for efficient and accurate protein language models},
  journal = {Bioinformatics},
  year = {2026},
  volume = {42},
  number = {Supplement_1},
  doi = {10.1093/bioinformatics/btag226}
}
Kostenlos starten →