DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey
PLOS ONE · 2023
DOI: 10.1371/journal.pone.0270616 ↗Kurz gefasst
Übersichtsarbeit in der Zeitschrift PLOS ONE, 2023. Verfasst von Anna R Gagliardi und 11 weiteren Autor:innen. Beteiligte Einrichtung: Toronto General Hospital Research Institute, University Health Network, Toronto, Ontario, Canada. Hauptthemen laut MEDLINE: Delivery of Health Care, Research Design. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Objective
The San Francisco Declaration on Research Assessment (DORA) advocates for assessing biomedical research quality and impact, yet academic organizations continue to employ traditional measures such as Journal Impact Factor. We aimed to identify and prioritize measures for assessing research quality and impact.
Methods
We conducted a review of published and grey literature to identify measures of research quality and impact, which we included in an online survey. We assembled a panel of researchers and research leaders, and conducted a two-round Delphi survey to prioritize measures rated as high (rated 6 or 7 by ≥ 80% of respondents) or moderate (rated 6 or 7 by ≥ 50% of respondents) importance.
Results
We identified 50 measures organized in 8 domains: relevance of the research program, challenges to research program, or productivity, team/open science, funding, innovations, publications, other dissemination, and impact. Rating of measures by 44 panelists (60%) in Round One and 24 (55%) in Round Two of a Delphi survey resulted in consensus on the high importance of 5 measures: research advances existing knowledge, research plan is innovative, an independent body of research (or fundamental role) supported by peer-reviewed research funding, research outputs relevant to discipline, and quality of the content of publications. Five measures achieved consensus on moderate importance: challenges to research productivity, potential to improve health or healthcare, team science, collaboration, and recognition by professional societies or academic bodies. There was high congruence between researchers and research leaders across disciplines.
Conclusions
Our work contributes to the field by identifying 10 DORA-compliant measures of research quality and impact, a more comprehensive and explicit set of measures than prior efforts. Research is needed to identify strategies to overcome barriers of use of DORA-compliant measures, and to “de-implement” traditional measures that do not uphold DORA principles yet are still in use.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Toronto General Hospital Research Institute, University Health Network, Toronto, Ontario, Canada.
- UHN Research Solutions and Services, University Health Network, Toronto, Ontario, Canada.
- The Institute for Education Research, University Health Network, Toronto, Ontario, Canada.
- Techna Institute, University Health Network, Toronto, Ontario, Canada.
- Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada.
- Toronto Rehabilitation Institute (KITE), University Health Network, Toronto, Ontario, Canada.
ORCID-Profile: Anna R Gagliardi, James Chow, Behrang Keshavarz
Zitieren
APA
Gagliardi, A. R., Chen, R. H. C., Boury, H., Albert, M., Chow, J., DaCosta, R. S., Hoffman, M., Keshavarz, B., Kontos, P., Liu, J., McAndrews, M. P., & Protze, S. (2023). DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey. PLOS ONE. https://doi.org/10.1371/journal.pone.0270616
Vancouver
Gagliardi AR, Chen RHC, Boury H, Albert M, Chow J, DaCosta RS, et al. DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey. PLOS ONE. 2023. doi:10.1371/journal.pone.0270616
BibTeX
@article{Gagliardi2023,
author = {Gagliardi, Anna R and Chen, Rob H C and Boury, Himani and Albert, Mathieu and Chow, James and DaCosta, Ralph S and Hoffman, Michael and Keshavarz, Behrang and Kontos, Pia and Liu, Jenny and McAndrews, Mary Pat and Protze, Stephanie},
title = {DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey},
journal = {PLOS ONE},
year = {2023},
doi = {10.1371/journal.pone.0270616}
}
EUVIMED