EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

A systematic assessment of single-cell language model configurations

Gaetan De Waele iD, Gerben Menschaert iD, Willem Waegeman iD

NAR Genomics and Bioinformatics · 27.06.2026 · 8 (3)

DOI: 10.1093/nargab/lqag095 ↗

Kurz gefasst

Fachartikel in der Zeitschrift NAR Genomics and Bioinformatics, online erschienen am 20.08.2026. Verfasst von Gaetan De Waele und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Data Analysis and Mathematical Modelling, Ghent University, Coupure Links 653, 9000, Belgium. Hauptthemen laut MEDLINE: Single-Cell Analysis. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Research Foundation - Flanders.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformers pre-trained on single-cell transcriptomic data have recently been applied to a series of tasks, earning them the title of foundation models. However, recent benchmarks indicate that current iterations of single-cell foundation models may often be surpassed by simpler task-specific models. As all currently published models in this class employ vastly different strategies, it is impossible to determine which practices drive their success (or failure). To help steer research in a more productive direction, we present a framework for the study of single-cell foundation models: bento-sc (BENchmarking Transformer-Obtained Single-Cell representations). We use bento-sc to perform a large-scale benchmarking of single-cell language model (scLM) configurations. By isolating parts of the pre-training scheme one by one, we define best practices for scLM construction. While comparisons with baselines indicate that scLMs do not yet offer the generational leap in prediction performances promised by many foundation models, we identify key design choices leading to improved performance. Namely, the best scLMs are obtained by: (i) minimally processing counts for input, (ii) using reconstruction losses that exploit known count distributions, (iii) masking (up to high rates), and (iv) combining different pre-training tasks/losses. All code supporting this study is distributed on PyPI and is packaged under: https://github.com/gdewael/bento-sc.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Single-Cell AnalysisHumansSingle-Cell Gene Expression AnalysisAlgorithms

Autor:innen und Institutionen

  • Department of Data Analysis and Mathematical Modelling, Ghent University, Coupure Links 653, 9000, Belgium.

ORCID-Profile: Gaetan De Waele, Gerben Menschaert, Willem Waegeman

Förderung

  • Research Foundation - Flanders

Zitieren

APA

De Waele, G., Menschaert, G., & Waegeman, W. (2026). A systematic assessment of single-cell language model configurations. NAR Genomics and Bioinformatics, 8(3). https://doi.org/10.1093/nargab/lqag095

Vancouver

De Waele G, Menschaert G, Waegeman W. A systematic assessment of single-cell language model configurations. NAR Genomics and Bioinformatics. 2026;8(3). doi:10.1093/nargab/lqag095

BibTeX
@article{DeWaele2026,
  author = {De Waele, Gaetan and Menschaert, Gerben and Waegeman, Willem},
  title = {A systematic assessment of single-cell language model configurations},
  journal = {NAR Genomics and Bioinformatics},
  year = {2026},
  volume = {8},
  number = {3},
  doi = {10.1093/nargab/lqag095}
}
Kostenlos starten →