TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling
Frontiers in Microbiology · 03.08.2026 · 17
DOI: 10.3389/fmicb.2026.1839420 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Microbiology, online erschienen am 03.08.2026. Verfasst von Fang Sheng und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China. Die Arbeit nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Microbial small proteins, encoded by small open reading frames (smORFs), play essential roles in antimicrobial activity, metabolic regulation, and signaling pathways. However, their short length and rapid evolutionary rate present significant challenges for computational modeling.
Methods
We introduce TinyProteinTransformer (TPT), a lightweight CNN-Transformer hybrid encoder pretrained on the Global Microbial smORF Catalog (GMSC; >280 million smORF families). TPT integrates multi-scale convolutional filters to capture local sequence motifs with Transformer layers for broader contextual modeling, and is trained jointly with masked language modeling and contrastive learning to capture both residue-level and sequence-level representations. We evaluated TPT on six downstream peptide/protein classification benchmarks spanning antimicrobial peptides (AMP), toxic peptides (TOX), bacteriocins (BCN), anti-CRISPR proteins (Acr), quorum-sensing peptides (QSP), and cell-penetrating peptides (CPP) using frozen-encoder linear probing.
Results
On the AMP and TOX tasks, TPT (103 M parameters) matched ESM2-150 M in predictive performance (AUC 0.930 vs. 0.928 for AMP; 0.930 vs. 0.925 for TOX) while achieving 4.3-fold faster inference. Across the evaluated benchmarks, TPT showed competitive performance compared with larger protein language models. Ablation experiments identified the contrastive objective as an important contributor to representation quality: its removal reduced AUC across all six downstream tasks and produced performance patterns consistent with MLM-only pretraining.
Conclusion
These results suggest that, within the evaluated benchmarks, effective smORF representation may benefit from pretraining objectives and inductive biases tailored to short, rapidly evolving sequences, rather than from model scale alone. TPT therefore provides a compact and computationally efficient encoder with competitive performance for microbial peptide analysis.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China.
- University of Toronto, Toronto, ON, Canada.
- Department of Thoracic Surgery, Zhongshan Hospital, Fudan University, Shanghai, China.
Zitieren
APA
Sheng, F., Zhang, J., & Zhu, C. (2026). TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling. Frontiers in Microbiology, 17. https://doi.org/10.3389/fmicb.2026.1839420
Vancouver
Sheng F, Zhang J, Zhu C. TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling. Frontiers in Microbiology. 2026;17. doi:10.3389/fmicb.2026.1839420
BibTeX
@article{Sheng2026,
author = {Sheng, Fang and Zhang, Junhe and Zhu, Chengkai},
title = {TPT: a compact CNN-transformer encoder for efficient microbial small protein modeling},
journal = {Frontiers in Microbiology},
year = {2026},
volume = {17},
doi = {10.3389/fmicb.2026.1839420}
}
EUVIMED