A systematic assessment of single-cell language model configurations
NAR Genomics and Bioinformatics · 27.06.2026 · 8 (3)
DOI: 10.1093/nargab/lqag095 ↗Kurz gefasst
Fachartikel in der Zeitschrift NAR Genomics and Bioinformatics, online erschienen am 20.08.2026. Verfasst von Gaetan De Waele und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Data Analysis and Mathematical Modelling, Ghent University, Coupure Links 653, 9000, Belgium. Hauptthemen laut MEDLINE: Single-Cell Analysis. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Research Foundation - Flanders.
Abstract (Originalsprache)
Transformers pre-trained on single-cell transcriptomic data have recently been applied to a series of tasks, earning them the title of foundation models. However, recent benchmarks indicate that current iterations of single-cell foundation models may often be surpassed by simpler task-specific models. As all currently published models in this class employ vastly different strategies, it is impossible to determine which practices drive their success (or failure). To help steer research in a more productive direction, we present a framework for the study of single-cell foundation models: bento-sc (BENchmarking Transformer-Obtained Single-Cell representations). We use bento-sc to perform a large-scale benchmarking of single-cell language model (scLM) configurations. By isolating parts of the pre-training scheme one by one, we define best practices for scLM construction. While comparisons with baselines indicate that scLMs do not yet offer the generational leap in prediction performances promised by many foundation models, we identify key design choices leading to improved performance. Namely, the best scLMs are obtained by: (i) minimally processing counts for input, (ii) using reconstruction losses that exploit known count distributions, (iii) masking (up to high rates), and (iv) combining different pre-training tasks/losses. All code supporting this study is distributed on PyPI and is packaged under: https://github.com/gdewael/bento-sc.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Data Analysis and Mathematical Modelling, Ghent University, Coupure Links 653, 9000, Belgium.
ORCID-Profile: Gaetan De Waele, Gerben Menschaert, Willem Waegeman
Förderung
- Research Foundation - Flanders
Zitieren
APA
De Waele, G., Menschaert, G., & Waegeman, W. (2026). A systematic assessment of single-cell language model configurations. NAR Genomics and Bioinformatics, 8(3). https://doi.org/10.1093/nargab/lqag095
Vancouver
De Waele G, Menschaert G, Waegeman W. A systematic assessment of single-cell language model configurations. NAR Genomics and Bioinformatics. 2026;8(3). doi:10.1093/nargab/lqag095
BibTeX
@article{DeWaele2026,
author = {De Waele, Gaetan and Menschaert, Gerben and Waegeman, Willem},
title = {A systematic assessment of single-cell language model configurations},
journal = {NAR Genomics and Bioinformatics},
year = {2026},
volume = {8},
number = {3},
doi = {10.1093/nargab/lqag095}
}
EUVIMED