EUVIMEDEUVIMED entdecken
Systematische ÜbersichtOpen Access · CC BY 4.0

Data extraction methods for systematic review (semi)automation: Update of a living systematic review

Lena Schmidt iD, Ailbhe N Finnerty Mutlu, Rebecca Elmore iD, Babatunde K Olorisade, James Thomas iD, Julian P T Higgins

F1000Research · 08.04.2025 · 10 · 401

DOI: 10.12688/f1000research.51117.3 ↗

Kurz gefasst

Systematische Übersicht in der Zeitschrift F1000Research, online erschienen am 08.04.2025. Verfasst von Lena Schmidt und 5 weiteren Autor:innen. Beteiligte Einrichtung: NIHR Innovation Observatory, Newcastle University, Newcastle upon Tyne, NE4 5TG, UK. Hauptthemen laut MEDLINE: Information Storage and Retrieval, Systematic Reviews as Topic. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 169 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Medical Research Council, National Institute for Health Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The reliable and usable (semi) automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies.

Methods

We systematically and continually search PubMed, ACL Anthology, arXiv, OpenAlex via EPPI-Reviewer, and the dblp computer science bibliography databases. Full text screening and data extraction are conducted using a mix of open-source and commercial tools. This living review update includes publications up to August 2024 and OpenAlex content up to September 2024.

Results

117 publications are included in this review. Of these, 30 (26%) used full texts while the rest used titles and abstracts. A total of 112 (96%) publications developed classifiers for randomised controlled trials. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. Data are available from 53 (45%), and code from 49 (42%) publications. Nine (8%) implemented publicly available tools.

Conclusions

This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of literature review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting other study types. Between review updates, large language models emerged as a new tool for data extraction. While facilitating access to automated extraction, they showed a trend of decreasing quality of results reporting, especially quantitative results such as recall and lower reproducibility of results. Compared with the previous update, trends such as transition to relation extraction and sharing of code and datasets stayed similar.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Information Storage and RetrievalSystematic Reviews as TopicHumansAutomation

Stichwörter der Autor:innen

Data ExtractionNatural Language ProcessingReproducibilitySystematic ReviewsText Mining

Autor:innen und Institutionen

  • NIHR Innovation Observatory, Newcastle University, Newcastle upon Tyne, NE4 5TG, UK.
  • Sciome LLC, Research Triangle Park, North Carolina, 27713, USA.
  • Bristol Medical School, University of Bristol, Bristol, BS8 2PS, UK.
  • UCL Social Research Institute, University College London, London, WC1H 0AL, UK.
  • Evaluate Ltd, London, SE1 2RE, UK.
  • Cardiff School of Technologies, Cardiff Metropolitan University, Cardiff, CF5 2YB, UK.

ORCID-Profile: Lena Schmidt, Rebecca Elmore, James Thomas

Förderung

  • Medical Research Council
  • National Institute for Health Research

Zitieren

APA

Schmidt, L., Finnerty Mutlu, A. N., Elmore, R., Olorisade, B. K., Thomas, J., & Higgins, J. P. T. (2025). Data extraction methods for systematic review (semi)automation: Update of a living systematic review. F1000Research, 10, 401. https://doi.org/10.12688/f1000research.51117.3

Vancouver

Schmidt L, Finnerty Mutlu AN, Elmore R, Olorisade BK, Thomas J, Higgins JPT. Data extraction methods for systematic review (semi)automation: Update of a living systematic review. F1000Research. 2025;10:401. doi:10.12688/f1000research.51117.3

BibTeX
@article{Schmidt2025,
  author = {Schmidt, Lena and Finnerty Mutlu, Ailbhe N and Elmore, Rebecca and Olorisade, Babatunde K and Thomas, James and Higgins, Julian P T},
  title = {Data extraction methods for systematic review (semi)automation: Update of a living systematic review},
  journal = {F1000Research},
  year = {2025},
  volume = {10},
  pages = {401},
  doi = {10.12688/f1000research.51117.3}
}
Kostenlos starten →