EUVIMEDOpdag EUVIMED
Scoping ReviewOpen access

Artificial Intelligence Tools for Automating Evidence Synthesis: Scoping Review

Künstliche Intelligenz zur Automatisierung der Evidenzsynthese: Scoping Review

Sashika Harasgama, Helen Pearce, Cameron Appel, Liam Loftus, Helena Painter, Isla Kuhn, Justine Karpusheff, Aji Ceesay, John Ford

Journal of Medical Internet Research · 2026-03-30 · 28 · e81597-e81597

DOI: 10.2196/81597 ↗

Auf einen Blick

Die Review kartiert evaluierte KI-Tools zur Automatisierung der Evidenzsynthese. 222 Artikel beschreiben 65 KI-Tools und 25 Open-Source-Modelle bzw. ML-Algorithmen.

Fragestellung

Welche evaluierten KI-Tools zur Automatisierung der Evidenzsynthese gibt es und wie werden sie untersucht?

Methode

Scoping Review nach Joanna-Briggs-Institute-Methodik; Suche in Ovid MEDLINE, Ovid Embase, Scopus und Web of Science im Februar 2025 sowie graue Literatur im April 2025.

Ergebnisse

222 eingeschlossene Artikel; 65 KI-Tools und 25 Open-Source-Modelle/ML-Algorithmen. 61,7 % untersuchten Titel-/Abstract-Screening, 26,1 % die Datenextraktion. 4,1 % berichteten Zeit- oder Arbeitsaufwandsdaten.

Einschränkungen

Keine Studien verglichen traditionelle ML-Tools mit LLM-basierten Tools; wenige pragmatische Real-World-Evaluationen.

Schlussfolgerung der Autor:innen

Die Review zeigt ein breites, sich rasch entwickelndes Spektrum an KI-Automatisierungstools für die Evidenzsynthese. Die fehlenden Vergleiche und pragmatischen Studien stellen eine wichtige Forschungslücke dar.

Kernaussagen

  • 222 Artikel wurden eingeschlossen
  • 61,7 % untersuchten Screening, 26,1 % die Datenextraktion
  • Nur 4,1 % berichteten zeit- oder arbeitsbezogene Outcomes

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Scoping Review in Journal of Medical Internet Research, published online on 2026-03-30. Written by Sashika Harasgama and 8 co-authors. Institution: Wolfson Institute of Population Health, Queen Mary University of London, Whitechapel Campus, London, E1 2AD, United Kingdom, +44 (0)20 7882 5555. Major MEDLINE topics: Artificial Intelligence. Cited 12 times so far (Crossref); 65 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Rapidly and accurately synthesizing large volumes of evidence is a time- and resource-intensive process. Once published, reviews often risk becoming outdated, limiting their usefulness for decision makers. Recent advancements in artificial intelligence (AI) have enabled researchers to automate stages of the evidence synthesis process, from literature searching and screening to data extraction and analysis. As previous reviews on this topic have been published, a significant number of tools have been further developed and evaluated. Furthermore, as generative AI increasingly automates evidence synthesis, understanding how it is studied and applied is crucial, given both its benefits and risks.

Objective

This review aimed to map the current landscape of evaluated AI tools used to automate evidence synthesis.

Methods

Following the Joanna Briggs Institute methodology for scoping reviews, we searched Ovid MEDLINE, Ovid Embase, Scopus, and Web of Science in February 2025 and conducted a gray literature search in April 2025. We included articles published in any language from January 2021 onward. Two reviewers independently screened citations using Rayyan, and data were extracted based on study design and key AI-related technical features.

Results

We identified 7841 unique citations through database searches and 19 records through gray literature searching. A total of 222 articles were included in the review. We identified 65 AI tools and 25 open-source models or machine learning (ML) algorithms that automate parts of or the whole evidence synthesis pathway. A total of 54.1% (n=120) of the studies were published in 2024, reflecting a trend toward researching general-purpose large language models (LLMs) for evidence synthesis automation. The most popular tool studied was generative pretrained transformer models, including its conversational interface ChatGPT (n=70, 31.5%). Moreover, 31.1% (n=69) studied tools automated by traditional ML algorithms. No studies compared traditional ML tools to LLM-based tools. In addition, 61.7% (n=137) and 26.1% (n=58) studied AI-assisted automation of title and abstract screening and data extraction, respectively, the 2 most intensive stages and, therefore, amenable to automation. Technical performance outcomes were the most frequently reported, with only 4.1% (n=9) of studies reporting time- or workload-specific outcomes. Few studies pragmatically evaluated AI tools in real-world evidence synthesis settings.

Conclusions

This review comprehensively captures the broad, evolving suite of AI automation tools available to support evidence synthesis, leveraged by increasingly complex AI approaches that range from traditional ML to LLMs. The notable shift toward studying general-purpose generative AI tools reflects how these technologies are actively transforming evidence synthesis practice. The lack of studies in our review comparing different AI approaches for specific automation stages or evaluating their effectiveness pragmatically represents a significant research gap. Optimal tool selection will likely depend on the review topic and methodology and researcher priorities. While they offer potential for reducing workload, ongoing evaluation to mitigate AI bias and to ensure the integrity of reviews is essential for safeguarding evidence-based decision-making.

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceGenerative Artificial IntelligenceLarge Language Models

Author keywords

ChatGPTartificial intelligenceautomationevidence synthesislarge language modelsmachine learningsystematic reviews as a topic

Authors and institutions

  • Wolfson Institute of Population Health, Queen Mary University of London, Whitechapel Campus, London, E1 2AD, United Kingdom, +44 (0)20 7882 5555.
  • Medical Library, University of Cambridge, Cambridge, United Kingdom.
  • The Health Foundation, London, United Kingdom.

Cite

APA

Harasgama, S., Pearce, H., Appel, C., Loftus, L., Painter, H., Kuhn, I., Karpusheff, J., Ceesay, A., & Ford, J. (2026). Artificial Intelligence Tools for Automating Evidence Synthesis: Scoping Review. Journal of Medical Internet Research, 28, e81597-e81597. https://doi.org/10.2196/81597

Vancouver

Harasgama S, Pearce H, Appel C, Loftus L, Painter H, Kuhn I, et al. Artificial Intelligence Tools for Automating Evidence Synthesis: Scoping Review. Journal of Medical Internet Research. 2026;28:e81597-e81597. doi:10.2196/81597

BibTeX
@article{Harasgama2026,
  author = {Harasgama, Sashika and Pearce, Helen and Appel, Cameron and Loftus, Liam and Painter, Helena and Kuhn, Isla and Karpusheff, Justine and Ceesay, Aji and Ford, John},
  title = {Artificial Intelligence Tools for Automating Evidence Synthesis: Scoping Review},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e81597--e81597},
  doi = {10.2196/81597}
}
Start for free →