EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling

Fang Sheng, Junhe Zhang, Chengkai Zhu

Frontiers in Microbiology · 03.08.2026 · 17

DOI: 10.3389/fmicb.2026.1839420 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Microbiology, online erschienen am 03.08.2026. Verfasst von Fang Sheng und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China. Die Arbeit nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Microbial small proteins, encoded by small open reading frames (smORFs), play essential roles in antimicrobial activity, metabolic regulation, and signaling pathways. However, their short length and rapid evolutionary rate present significant challenges for computational modeling.

Methods

We introduce TinyProteinTransformer (TPT), a lightweight CNN-Transformer hybrid encoder pretrained on the Global Microbial smORF Catalog (GMSC; >280 million smORF families). TPT integrates multi-scale convolutional filters to capture local sequence motifs with Transformer layers for broader contextual modeling, and is trained jointly with masked language modeling and contrastive learning to capture both residue-level and sequence-level representations. We evaluated TPT on six downstream peptide/protein classification benchmarks spanning antimicrobial peptides (AMP), toxic peptides (TOX), bacteriocins (BCN), anti-CRISPR proteins (Acr), quorum-sensing peptides (QSP), and cell-penetrating peptides (CPP) using frozen-encoder linear probing.

Results

On the AMP and TOX tasks, TPT (103 M parameters) matched ESM2-150 M in predictive performance (AUC 0.930 vs. 0.928 for AMP; 0.930 vs. 0.925 for TOX) while achieving 4.3-fold faster inference. Across the evaluated benchmarks, TPT showed competitive performance compared with larger protein language models. Ablation experiments identified the contrastive objective as an important contributor to representation quality: its removal reduced AUC across all six downstream tasks and produced performance patterns consistent with MLM-only pretraining.

Conclusion

These results suggest that, within the evaluated benchmarks, effective smORF representation may benefit from pretraining objectives and inductive biases tailored to short, rapidly evolving sequences, rather than from model scale alone. TPT therefore provides a compact and computationally efficient encoder with competitive performance for microbial peptide analysis.

Schlagwörter

Stichwörter der Autor:innen

deep learningmicroproteinspeptide predictionprotein language modelsmORFs

Autor:innen und Institutionen

  • Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China.
  • University of Toronto, Toronto, ON, Canada.
  • Department of Thoracic Surgery, Zhongshan Hospital, Fudan University, Shanghai, China.

Zitieren

APA

Sheng, F., Zhang, J., & Zhu, C. (2026). TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling. Frontiers in Microbiology, 17. https://doi.org/10.3389/fmicb.2026.1839420

Vancouver

Sheng F, Zhang J, Zhu C. TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling. Frontiers in Microbiology. 2026;17. doi:10.3389/fmicb.2026.1839420

BibTeX
@article{Sheng2026,
  author = {Sheng, Fang and Zhang, Junhe and Zhu, Chengkai},
  title = {TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling},
  journal = {Frontiers in Microbiology},
  year = {2026},
  volume = {17},
  doi = {10.3389/fmicb.2026.1839420}
}
Kostenlos starten →