EUVIMEDEUVIMED entdecken
Open Access

LitAutoScreener: Development and Validation of an Automated Literature Screening Tool in Evidence-Based Medicine Driven by Large Language Models

Yiming Tao iD, Xuehu Li, Zuhar Yisha, Sihan Yang, Siyan Zhan, Feng Sun iD

Health Data Science · 2025-01 · 5

DOI: 10.34133/hds.0322 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Health Data Science, online erschienen am 02.09.2025. Verfasst von Yiming Tao und 5 weiteren Autor:innen. Beteiligte Einrichtung: Key Laboratory of Epidemiology of Major Diseases, Ministry of Education/Department of Epidemiology and Biostatistics, School of Public Health, Peking…. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: National Natural Science Foundation of China, the third batches of Key Projects of Scientific Act for Drug Regulation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The traditional manual literature screening approach is limited by its time-consuming nature and high labor costs. A pressing issue is how to leverage large language models to enhance the efficiency and quality of evidence-based evaluations of drug efficacy and safety.

Methods

This study utilized a manually curated reference literature database—comprising vaccine, hypoglycemic agent, and antidepressant evaluation studies—previously developed by our team through conventional systematic review methods. This validated database served as the gold standard for the development and optimization of LitAutoScreener. Following the PICOS (Population, Intervention, Comparison, Outcomes, Study Design) principles, a chain-of-thought reasoning approach with few-shot learning prompts was implemented to develop the screening algorithm. We subsequently evaluated the performance of LitAutoScreener using 2 independent validation cohorts, assessing both classification accuracy and processing efficiency.

Results

For respiratory syncytial virus vaccine safety validation title–abstract screening, our tools based on GPT (GPT-4o), Kimi (moonshot-v1-128k), and DeepSeek (deepseek-chat 2.5) demonstrated high accuracy in inclusion/exclusion decisions (99.38%, 98.94%, and 98.85%, respectively). Recall rates were 100.00%, 99.13%, and 98.26%, with statistically significant performance differences ( χ 2 = 5.99, P = 0.048), where GPT outperformed the other models. Exclusion reason concordance rates were 98.85%, 94.79%, and 96.47% ( χ 2 = 30.22, P < 0.001). In full-text screening, all models maintained perfect recall (100.00%), with accuracies of 100.00% (GPT), 100.00% (Kimi), and 99.45% (DeepSeek). Processing times averaged 1 to 5 s per article for title–abstract screening and 60 s for full-text processing (including PDF preprocessing).

Conclusions

LitAutoScreener offers a new approach for efficient literature screening in drug intervention studies, achieving high accuracy and significantly improving screening efficiency.

Autor:innen und Institutionen

  • Key Laboratory of Epidemiology of Major Diseases, Ministry of Education/Department of Epidemiology and Biostatistics, School of Public Health, Peking University, Beijing, China.
  • School of Cybersecurity, Hainan University, Hainan, China.
  • School of Public Health, Peking University, Beijing, China.
  • Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China.
  • Center for Intelligent Public Health, Institute for Artificial Intelligence, Peking University, Beijing, China.
  • Xinjiang Medical University Institute of Traditional Chinese Medicine, Urumqi 830017, China.

ORCID-Profile: Yiming Tao, Feng Sun

Förderung

  • National Natural Science Foundation of China
  • the third batches of Key Projects of Scientific Act for Drug Regulation of China

Zitieren

APA

Tao, Y., Li, X., Yisha, Z., Yang, S., Zhan, S., & Sun, F. (2025). LitAutoScreener: Development and Validation of an Automated Literature Screening Tool in Evidence-Based Medicine Driven by Large Language Models. Health Data Science, 5. https://doi.org/10.34133/hds.0322

Vancouver

Tao Y, Li X, Yisha Z, Yang S, Zhan S, Sun F. LitAutoScreener: Development and Validation of an Automated Literature Screening Tool in Evidence-Based Medicine Driven by Large Language Models. Health Data Science. 2025;5. doi:10.34133/hds.0322

BibTeX
@article{Tao2025,
  author = {Tao, Yiming and Li, Xuehu and Yisha, Zuhar and Yang, Sihan and Zhan, Siyan and Sun, Feng},
  title = {LitAutoScreener: Development and Validation of an Automated Literature Screening Tool in Evidence-Based Medicine Driven by Large Language Models},
  journal = {Health Data Science},
  year = {2025},
  volume = {5},
  doi = {10.34133/hds.0322}
}
Kostenlos starten →