Benchmarking large language models for identifying transcription factor regulatory interactions
Bioinformatics · 2026
DOI: 10.1093/bioinformatics/btaf653 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics, 2026. Verfasst von Lake Noel und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States. Hauptthemen laut MEDLINE: Transcription Factors, Computational Biology, Gene Regulatory Networks, Programming Languages. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Glazer Foundation Award, NIH HHS, NCI NIH HHS.
Abstract (Originalsprache)
Motivation Transcription factors (TFs) and their target genes form regulatory networks that control gene expression and influence diverse biological processes and disease outcomes. Although multiple computational methods and curated databases have been developed to identify TF–target interactions, they often require specialized expertise. Large language models (LLMs) chatbots offer a more accessible alternative for querying TF–target interactions. In this study, we benchmarked four prominent LLMs, Anthropic’s Claude 3.5 Sonnet, Google’s Gemini 1.0 Pro, OpenAI’s GPT-4o, and Meta’s Llama3 8b, using 8432 literature-curated human TF–target interactions. We examined four regulatory categories: bidirectional, ambiguous, self-regulated, and unidirectional interactions. Results Under single-turn queries, Claude 3.5 Sonnet and GPT-4o outperformed the others, with balanced accuracies reaching 50.0 ± 7.6% (GPT-4o, self-regulated) and 48.2 ± 1.0% (Claude 3.5 Sonnet, unidirectional). Zero-temperature settings generally enhanced reproducibility, and multi-turn prompting improved performance for most models, increasing Claude 3.5 Sonnet’s accuracy on self-regulated pairs by 32.6%. Excluding TF–target pairs with all unknown regulation types also generally improved accuracy, with unidirectional regulation reaching near 70% balanced accuracy in some cases. We also benchmarked Anthropic’s Claude 3.5 Sonnet, Google’s Gemini 2.0 Flash, OpenAI’s GPT-4o, and Meta’s Llama3 using 5148 experimentally derived TF–target interactions. Claude 3.5 Sonnet consistently outperformed the other models across conditions. Our findings highlight that prompt engineering and strategic use of model parameters consistently influence LLM chatbots’ performance on TF–target identifications. This study establishes a benchmarking framework and demonstrates the potential of pre-trained general-purpose LLMs to support regulatory biology research, especially for researchers without extensive computational expertise. Availability and implementation The literature-based TF–target interactions ground truth were obtained from TRRUST v2 human dataset (www.grnpedia.org/trrust). The experimental derived TF–target interactions ground truth were obtained from TFLink Home Sapiens small-scale interaction table (https://tflink.net/). Processed TF–target interactions data and the analytical pipeline has been compiled as an interactive Python notebook file and is available at https://github.com/pengpclab/LLM-TF-interactions.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Substanzen
Autor:innen und Institutionen
- Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States.
- Biomedical Imaging Research Institute, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States.
- Department of Urology, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States.
- Department of Surgery, Cedars-Sinai Medical Center, Los Angeles, CA 90048, United States.
ORCID-Profile: Pei-Chen Peng, Xiuzhen Huang
Förderung
- Glazer Foundation Award
- NIH HHS
- NCI NIH HHS
- Jim and Eleanor Department of Surgery Research Award
- NIA NIH HHS
- Samuel Oschin Cancer Institute Research Development Fund
- National Institutes of Health
Zitieren
APA
Noel, L., Hsiao, Y. W., He, Y., Hung, A., Cui, X., Ray, E., Moore, J. H., Peng, P. C., & Huang, X. (2026). Benchmarking large language models for identifying transcription factor regulatory interactions. Bioinformatics. https://doi.org/10.1093/bioinformatics/btaf653
Vancouver
Noel L, Hsiao YW, He Y, Hung A, Cui X, Ray E, et al. Benchmarking large language models for identifying transcription factor regulatory interactions. Bioinformatics. 2026. doi:10.1093/bioinformatics/btaf653
BibTeX
@article{Noel2026,
author = {Noel, Lake and Hsiao, Yi-Wen and He, Yimeng and Hung, Andrew and Cui, Xiaojiang and Ray, Edward and Moore, Jason H and Peng, Pei-Chen and Huang, Xiuzhen},
title = {Benchmarking large language models for identifying transcription factor regulatory interactions},
journal = {Bioinformatics},
year = {2026},
doi = {10.1093/bioinformatics/btaf653}
}
EUVIMED