EUVIMEDEUVIMED entdecken
Open Access

Confidence score: a data-driven measure for inclusive systematic reviews considering unpublished preprints

Jiayi Tong, Chongliang Luo, Yifei Sun, Rui Duan iD, M Elle Saine, Lifeng Lin, Yifan Peng iD, Yiwen Lu, Anchita Batra, Anni Pan, Olivia Wang, Ruowang Li und 11 weitere

Journal of the American Medical Informatics Association · 03.04.2024 · 31 (4) · 809-819

DOI: 10.1093/jamia/ocad248 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 08.12.2023. Verfasst von Jiayi Tong und 22 weiteren Autor:innen. Beteiligte Einrichtung: The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylv…. Hauptthemen laut MEDLINE: COVID-19. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NCATS NIH HHS, NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

COVID-19, since its emergence in December 2019, has globally impacted research. Over 360 000 COVID-19-related manuscripts have been published on PubMed and preprint servers like medRxiv and bioRxiv, with preprints comprising about 15% of all manuscripts. Yet, the role and impact of preprints on COVID-19 research and evidence synthesis remain uncertain. Materials and Methods We propose a novel data-driven method for assigning weights to individual preprints in systematic reviews and meta-analyses. This weight termed the “confidence score” is obtained using the survival cure model, also known as the survival mixture model, which takes into account the time elapsed between posting and publication of a preprint, as well as metadata such as the number of first 2-week citations, sample size, and study type.

Results

Using 146 preprints on COVID-19 therapeutics posted from the beginning of the pandemic through April 30, 2021, we validated the confidence scores, showing an area under the curve of 0.95 (95% CI, 0.92-0.98). Through a use case on the effectiveness of hydroxychloroquine, we demonstrated how these scores can be incorporated practically into meta-analyses to properly weigh preprints.

Discussion

It is important to note that our method does not aim to replace existing measures of study quality but rather serves as a supplementary measure that overcomes some limitations of current approaches.

Conclusion

Our proposed confidence score has the potential to improve systematic reviews of evidence related to COVID-19 and other clinical conditions by providing a data-driven approach to including unpublished manuscripts.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

COVID-19HumansPandemicsPubMedResearch DesignSystematic Reviews as TopicMeta-Analysis as TopicPreprints as Topic

Stichwörter der Autor:innen

data-driven modelingevidence synthesispreprintsystematic review

Autor:innen und Institutionen

  • The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, PA 19104, United States.
  • Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, The University of Pennsylvania, Philadelphia, PA 19104, United States.
  • Division of Public Health Sciences, Washington University School of Medicine in St Louis, St Louis, MO 63110, United States.
  • Department of Biostatistics, Columbia University, New York City, NY 10032, United States.
  • Department of Biostatistics, Harvard T.H. Chan School of Public Health, Harvard University, Cambridge, MA 02115, United States.
  • Department of Epidemiology and Biostatistics, University of Arizona, Tucson, AZ 85724, United States.

ORCID-Profile: Rui Duan, Yifan Peng, Jiang Bian, Yong Chen

Förderung

  • NIA NIH HHS
  • NCATS NIH HHS
  • NIH HHS
  • NLM NIH HHS
  • Patient-Centered Outcomes Research Institute
  • NIAID NIH HHS
  • NIGMS NIH HHS
  • NEI NIH HHS

Zitieren

APA

Tong, J., Luo, C., Sun, Y., Duan, R., Saine, M. E., Lin, L., Peng, Y., Lu, Y., Batra, A., Pan, A., Wang, O., Li, R., Marks-Anglin, A., Yang, Y., Zuo, X., Liu, Y., Bian, J., Kimmel, S. E., Hamilton, K., …, & Chen, Y. (2024). Confidence score: a data-driven measure for inclusive systematic reviews considering unpublished preprints. Journal of the American Medical Informatics Association, 31(4), 809-819. https://doi.org/10.1093/jamia/ocad248

Vancouver

Tong J, Luo C, Sun Y, Duan R, Saine ME, Lin L, et al. Confidence score: a data-driven measure for inclusive systematic reviews considering unpublished preprints. Journal of the American Medical Informatics Association. 2024;31(4):809-819. doi:10.1093/jamia/ocad248

BibTeX
@article{Tong2024,
  author = {Tong, Jiayi and Luo, Chongliang and Sun, Yifei and Duan, Rui and Saine, M Elle and Lin, Lifeng and Peng, Yifan and Lu, Yiwen and Batra, Anchita and Pan, Anni and Wang, Olivia and Li, Ruowang and Marks-Anglin, Arielle and Yang, Yuchen and Zuo, Xu and Liu, Yulun and Bian, Jiang and Kimmel, Stephen E and Hamilton, Keith and Cuker, Adam and Hubbard, Rebecca A and Xu, Hua and Chen, Yong},
  title = {Confidence score: a data-driven measure for inclusive systematic reviews considering unpublished preprints},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  volume = {31},
  number = {4},
  pages = {809--819},
  doi = {10.1093/jamia/ocad248}
}
Kostenlos starten →