EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating AI Performance in Systematic Literature Reviews for HEOR: A Case Study

Jing Wang-Silvanto iD, Mansee Jajoo, He Guo, Rishi Ohri, Judith Nelissen, Carolina Casañas I Comabella

Journal of Health Economics and Outcomes Research · 07.08.2026 · 13 (2)

DOI: 10.36469/001c.165173 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Health Economics and Outcomes Research, online erschienen am 07.08.2026. Verfasst von Jing Wang-Silvanto und 5 weiteren Autor:innen. Beteiligte Einrichtung: Astellas Pharma A/S, filial I Finland, Espoo; Observational Health Data Sciences and Informatics (OHDSI), Finland. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Systematic literature reviews (SLRs) are central to evidence generation in health economics and outcomes research, but traditional SLR workflows are labor-intensive. Artificial intelligence (AI) is increasingly being explored to support evidence synthesis tasks.

Objectives

To describe methods used to assess AI performance in screening and data extraction in SLRs, and to provide recommendations on how to ensure appropriate use of AI in literature reviews.

Methods

We conducted an AI-assisted SLR that mirrored a traditional SLR performed by humans only and assessed the performance of the AI tools employed. AI was used to screen titles/abstracts, screen full texts, and conduct data extraction. Using the traditional SLR as the benchmark, AI performance for title/abstract screening was evaluated in terms of accuracy, recall, and precision at predefined screening milestones, followed by the accuracy assessment of full-text screening by AI. Data items extracted by AI were verified by humans and categorized as correct, incomplete, missing, incorrect, or requiring human checks. The average accuracy rate calculated on the basis of correct data items extracted was used as an indicator of AI performance in data extraction.

Results

During title/abstract screening, accuracy and recall remained high but precision was low, increasing false positives. Full-text screening identified a minority of studies included in the traditional SLR. Mean extraction accuracy was 72.93% (range, 57.69%-88.46%). No data were hallucinated by the AI model used.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligenceautomation toolsevidence synthesislarge language modelsmachine learningsystematic literature reviews

Autor:innen und Institutionen

  • Astellas Pharma A/S, filial I Finland, Espoo; Observational Health Data Sciences and Informatics (OHDSI), Finland.
  • Astellas Pharma, Astellas Pharma Global Development Inc. Northbrook, Illinois, USA.
  • Astellas Pharma Europe BV, Leiden, The Netherlands.
  • PPD Evidera Health Economics & Market Access, Thermo Fisher Scientific, London, UK.

ORCID-Profile: Jing Wang-Silvanto

Zitieren

APA

Wang-Silvanto, J., Jajoo, M., Guo, H., Ohri, R., Nelissen, J., & Casañas I Comabella, C. (2026). Evaluating AI Performance in Systematic Literature Reviews for HEOR: A Case Study. Journal of Health Economics and Outcomes Research, 13(2). https://doi.org/10.36469/001c.165173

Vancouver

Wang-Silvanto J, Jajoo M, Guo H, Ohri R, Nelissen J, Casañas I Comabella C. Evaluating AI Performance in Systematic Literature Reviews for HEOR: A Case Study. Journal of Health Economics and Outcomes Research. 2026;13(2). doi:10.36469/001c.165173

BibTeX
@article{WangSilvanto2026,
  author = {Wang-Silvanto, Jing and Jajoo, Mansee and Guo, He and Ohri, Rishi and Nelissen, Judith and Casañas I Comabella, Carolina},
  title = {Evaluating AI Performance in Systematic Literature Reviews for HEOR: A Case Study},
  journal = {Journal of Health Economics and Outcomes Research},
  year = {2026},
  volume = {13},
  number = {2},
  doi = {10.36469/001c.165173}
}
Kostenlos starten →