EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

FragBERTa: a fragment-aware molecular representation model with sequential attachment-based fragment embeddings

Neerav Kaushal, Ajay M N V Penmatsa

Machine Learning: Science and Technology · 01.08.2026 · 7 (4) · 045046

DOI: 10.1088/2632-2153/ae911d ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Learning: Science and Technology, online erschienen am 07.08.2026. Verfasst von Neerav Kaushal und 1 weiteren Autor:innen. Die Arbeit nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Nvidia, National Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automated exploration of chemical space through machine learning has become a central strategy in modern drug discovery, where the ability to learn transferable molecular representations directly impacts predictive modeling performance. Recent advances in chemical language models have primarily relied on atom-level string encodings such as SMILES or SELFIES, treating molecules as sequences of individual atoms or characters. However, medicinal chemistry reasoning is often fragment-centric, emphasizing scaffolds, functional groups, and modular substructures rather than isolated atoms. In this study, we introduce FragBERTa, a molecular fragment-aware transformer-based representation language model pretrained using masked language modeling on Sequential Attachment-based Fragment Embedding (SAFE) representations. SAFE decomposes molecules into chemically meaningful fragments while preserving attachment information, enabling the model to learn fragment-level structural patterns within a sequence framework. FragBERTa is pretrained on 11 million unique SAFE sequences extracted from multiple public chemical databases and subsequently finetuned on diverse MoleculeNet benchmarks covering both regression (ESOL, FreeSolv, Lipophilicity, PDBbind) and classification (BACE, BBBP, HIV, Tox21, SIDER) tasks under both random and scaffold splits. Our evaluation shows that FragBERTa achieves competitive performance across tasks, often matching or surpassing prior models. In regression, it demonstrates strong gains in lipophilicity and the best scaffold-split performance on protein-ligand binding affinity within our comparison. In classification, FragBERTa attains the strongest reported random-split ROC-AUC results on BACE, HIV, and Tox21 and the strongest reported scaffold-split ROC-AUC performance on BBBP among the compared baselines, and delivers robust performance across bioactivity, permeability, and toxicity benchmarks, indicating that fragment-based representations effectively capture biologically driven and interaction-sensitive endpoints. Collectively, these findings suggest that fragment-based string representations offer advantages over atom-level representations such as SMILES and SELFIES for scaffold-sensitive and interaction-driven tasks, highlighting the value of fragment-level inductive bias for future molecular foundation models in drug discovery.

Förderung

  • Nvidia
  • National Science Foundation

Zitieren

APA

Kaushal, N., & Penmatsa, A. M. N. V. (2026). FragBERTa: a fragment-aware molecular representation model with sequential attachment-based fragment embeddings. Machine Learning: Science and Technology, 7(4), 045046. https://doi.org/10.1088/2632-2153/ae911d

Vancouver

Kaushal N, Penmatsa AMNV. FragBERTa: a fragment-aware molecular representation model with sequential attachment-based fragment embeddings. Machine Learning: Science and Technology. 2026;7(4):045046. doi:10.1088/2632-2153/ae911d

BibTeX
@article{Kaushal2026,
  author = {Kaushal, Neerav and Penmatsa, Ajay M N V},
  title = {FragBERTa: a fragment-aware molecular representation model with sequential attachment-based fragment embeddings},
  journal = {Machine Learning: Science and Technology},
  year = {2026},
  volume = {7},
  number = {4},
  pages = {045046},
  doi = {10.1088/2632-2153/ae911d}
}
Kostenlos starten →