EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Benchmarking AI scientists for omics data–driven biological discovery

Erpai Luo iD, Jinmeng Jia, Yifan Xiong, Xiangyu Li iD, Xiaobo Guo, Baoqi Yu, Minsheng Hao, Lei Wei, Xuegong Zhang iD

Bioinformatics · 01.07.2026 · 42 (Supplement_1)

DOI: 10.1093/bioinformatics/btag227 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 07.07.2026. Verfasst von Erpai Luo und 8 weiteren Autor:innen. Beteiligte Einrichtung: MOE Key Laboratory of Bioinformatics and Bioinformatics Division of BNRIST, Department of Automation, Tsinghua University, Beijing, 100084, China. Hauptthemen laut MEDLINE: Benchmarking, Artificial Intelligence, Computational Biology. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, National Key R&D Program of China, Tsinghua-Toyota Joint Research Institute Inter-disciplinary Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation Recent advances in large language models have enabled the emergence of AI scientists that aim to autonomously analyze biological data and assist scientific discovery. Despite rapid progress, it remains unclear to what extent these systems can extract meaningful biological insights from real experimental data. Existing benchmarks either evaluate reasoning in the absence of data or focus on predefined analytical outputs, failing to reflect realistic, data-driven biological research. Results Here, we introduce BAISBench (Biological AI Scientist Benchmark), a benchmark for evaluating AI scientists on real single-cell transcriptomic datasets. BAISBench comprises two tasks: cell type annotation across 15 expert-labeled datasets, and scientific discovery through 193 multiple-choice questions derived from biological conclusions reported in 41 published single-cell studies. We evaluated several representative AI scientists using BAISBench and, to provide a human performance baseline, invited five graduate-level bioinformaticians to collectively complete the same tasks. The results show that while current AI scientists fall short of fully autonomous biological discovery, they already demonstrate substantial potential in supporting data-driven biological research. These results position BAISBench as a practical benchmark for characterizing the current capabilities and limitations of AI scientists in biological research. We expect BAISBench to serve as a practical evaluation framework for guiding the development of more capable AI scientists and for helping biologists identify AI systems that can effectively support real-world research workflows. Availability and implementation https://github.com/EperLuo/BAISBench, https://huggingface.co/datasets/EperLuo/BaisBench.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BenchmarkingArtificial IntelligenceComputational BiologyHumansLarge Language ModelsSingle-Cell AnalysisIntelligent Systems

Autor:innen und Institutionen

  • MOE Key Laboratory of Bioinformatics and Bioinformatics Division of BNRIST, Department of Automation, Tsinghua University, Beijing, 100084, China.
  • School of Software Engineering, Beijing Jiaotong University, Beijing, 100044, China.
  • Department of Physiology and Pathophysiology, School of Basic Medical Sciences, Capital Medical University, Beijing, 100069, China.
  • Center for Synthetic and Systems Biology, School of Life Sciences and School of Medicine, Tsinghua University, Beijing, 100084, China.

ORCID-Profile: Erpai Luo, Xiangyu Li, Xuegong Zhang

Förderung

  • National Natural Science Foundation of China
  • National Key R&D Program of China
  • Tsinghua-Toyota Joint Research Institute Inter-disciplinary Program

Zitieren

APA

Luo, E., Jia, J., Xiong, Y., Li, X., Guo, X., Yu, B., Hao, M., Wei, L., & Zhang, X. (2026). Benchmarking AI scientists for omics data–driven biological discovery. Bioinformatics, 42(Supplement_1). https://doi.org/10.1093/bioinformatics/btag227

Vancouver

Luo E, Jia J, Xiong Y, Li X, Guo X, Yu B, et al. Benchmarking AI scientists for omics data–driven biological discovery. Bioinformatics. 2026;42(Supplement_1). doi:10.1093/bioinformatics/btag227

BibTeX
@article{Luo2026,
  author = {Luo, Erpai and Jia, Jinmeng and Xiong, Yifan and Li, Xiangyu and Guo, Xiaobo and Yu, Baoqi and Hao, Minsheng and Wei, Lei and Zhang, Xuegong},
  title = {Benchmarking AI scientists for omics data–driven biological discovery},
  journal = {Bioinformatics},
  year = {2026},
  volume = {42},
  number = {Supplement_1},
  doi = {10.1093/bioinformatics/btag227}
}
Kostenlos starten →