EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Predicting the sample size of randomized controlled trials using natural language processing

Paul Windisch iD, Fabio Dennstädt, Carole Koechli, Robert Förster, Christina Schröder, Daniel M Aebersold, Daniel R Zwahlen

JAMIA Open · 08.10.2024 · 7 (4)

DOI: 10.1093/jamiaopen/ooae116 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 25.10.2024. Verfasst von Paul Windisch und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiation Oncology, Cantonal Hospital Winterthur, 8400 Winterthur, Switzerland. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Extracting the sample size from randomized controlled trials (RCTs) remains a challenge to developing better search functionalities or automating systematic reviews. Most current approaches rely on the sample size being explicitly mentioned in the abstract. The objective of this study was, therefore, to develop and validate additional approaches. Materials and Methods 847 RCTs from high-impact medical journals were tagged with 6 different entities that could indicate the sample size. A named entity recognition (NER) model was trained to extract the entities and then deployed on a test set of 150 RCTs. The entities’ performance in predicting the actual number of trial participants who were randomized was assessed and possible combinations of the entities were evaluated to create predictive models. The test set was also used to evaluate the performance of GPT-4o on the same task.

Results

The most accurate model could make predictions for 64.7% of trials in the test set, and the resulting predictions were equal to the ground truth in 93.8%. GPT-4o was able to make a prediction on 94.7% of trials and the resulting predictions were equal to the ground truth in 90.8%.

Discussion

This study presents an NER model that can extract different entities that can be used to predict the sample size from the abstract of an RCT. The entities can be combined in different ways to obtain models with different characteristics.

Conclusion

Training an NER model to predict the sample size from RCTs is feasible. Large language models can deliver similar performance without the need for prior training on the task although at a higher cost due to proprietary technology and/or required computational power.

Schlagwörter

Stichwörter der Autor:innen

GPT-4evidence-based medicinemachine learningnatural language processingrandomized controlled trialtext miningtransformer

Autor:innen und Institutionen

  • Department of Radiation Oncology, Cantonal Hospital Winterthur, 8400 Winterthur, Switzerland.
  • Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern, 3012 Bern, Switzerland.

ORCID-Profile: Paul Windisch

Zitieren

APA

Windisch, P., Dennstädt, F., Koechli, C., Förster, R., Schröder, C., Aebersold, D. M., & Zwahlen, D. R. (2024). Predicting the sample size of randomized controlled trials using natural language processing. JAMIA Open, 7(4). https://doi.org/10.1093/jamiaopen/ooae116

Vancouver

Windisch P, Dennstädt F, Koechli C, Förster R, Schröder C, Aebersold DM, et al. Predicting the sample size of randomized controlled trials using natural language processing. JAMIA Open. 2024;7(4). doi:10.1093/jamiaopen/ooae116

BibTeX
@article{Windisch2024,
  author = {Windisch, Paul and Dennstädt, Fabio and Koechli, Carole and Förster, Robert and Schröder, Christina and Aebersold, Daniel M and Zwahlen, Daniel R},
  title = {Predicting the sample size of randomized controlled trials using natural language processing},
  journal = {JAMIA Open},
  year = {2024},
  volume = {7},
  number = {4},
  doi = {10.1093/jamiaopen/ooae116}
}
Kostenlos starten →