EUVIMEDEUVIMED entdecken
ÜbersichtsarbeitDelphi-StudieOpen Access · CC BY 4.0

DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey

Anna R Gagliardi iD, Rob H C Chen, Himani Boury, Mathieu Albert, James Chow iD, Ralph S DaCosta, Michael Hoffman, Behrang Keshavarz iD, Pia Kontos, Jenny Liu, Mary Pat McAndrews, Stephanie Protze

PLOS ONE · 2023

DOI: 10.1371/journal.pone.0270616 ↗

Kurz gefasst

Übersichtsarbeit in der Zeitschrift PLOS ONE, 2023. Verfasst von Anna R Gagliardi und 11 weiteren Autor:innen. Beteiligte Einrichtung: Toronto General Hospital Research Institute, University Health Network, Toronto, Ontario, Canada. Hauptthemen laut MEDLINE: Delivery of Health Care, Research Design. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

The San Francisco Declaration on Research Assessment (DORA) advocates for assessing biomedical research quality and impact, yet academic organizations continue to employ traditional measures such as Journal Impact Factor. We aimed to identify and prioritize measures for assessing research quality and impact.

Methods

We conducted a review of published and grey literature to identify measures of research quality and impact, which we included in an online survey. We assembled a panel of researchers and research leaders, and conducted a two-round Delphi survey to prioritize measures rated as high (rated 6 or 7 by ≥ 80% of respondents) or moderate (rated 6 or 7 by ≥ 50% of respondents) importance.

Results

We identified 50 measures organized in 8 domains: relevance of the research program, challenges to research program, or productivity, team/open science, funding, innovations, publications, other dissemination, and impact. Rating of measures by 44 panelists (60%) in Round One and 24 (55%) in Round Two of a Delphi survey resulted in consensus on the high importance of 5 measures: research advances existing knowledge, research plan is innovative, an independent body of research (or fundamental role) supported by peer-reviewed research funding, research outputs relevant to discipline, and quality of the content of publications. Five measures achieved consensus on moderate importance: challenges to research productivity, potential to improve health or healthcare, team science, collaboration, and recognition by professional societies or academic bodies. There was high congruence between researchers and research leaders across disciplines.

Conclusions

Our work contributes to the field by identifying 10 DORA-compliant measures of research quality and impact, a more comprehensive and explicit set of measures than prior efforts. Research is needed to identify strategies to overcome barriers of use of DORA-compliant measures, and to “de-implement” traditional measures that do not uphold DORA principles yet are still in use.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Delivery of Health CareResearch DesignConsensusJournal Impact FactorSurveys and QuestionnairesDelphi Technique

Autor:innen und Institutionen

  • Toronto General Hospital Research Institute, University Health Network, Toronto, Ontario, Canada.
  • UHN Research Solutions and Services, University Health Network, Toronto, Ontario, Canada.
  • The Institute for Education Research, University Health Network, Toronto, Ontario, Canada.
  • Techna Institute, University Health Network, Toronto, Ontario, Canada.
  • Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada.
  • Toronto Rehabilitation Institute (KITE), University Health Network, Toronto, Ontario, Canada.

ORCID-Profile: Anna R Gagliardi, James Chow, Behrang Keshavarz

Zitieren

APA

Gagliardi, A. R., Chen, R. H. C., Boury, H., Albert, M., Chow, J., DaCosta, R. S., Hoffman, M., Keshavarz, B., Kontos, P., Liu, J., McAndrews, M. P., & Protze, S. (2023). DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey. PLOS ONE. https://doi.org/10.1371/journal.pone.0270616

Vancouver

Gagliardi AR, Chen RHC, Boury H, Albert M, Chow J, DaCosta RS, et al. DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey. PLOS ONE. 2023. doi:10.1371/journal.pone.0270616

BibTeX
@article{Gagliardi2023,
  author = {Gagliardi, Anna R and Chen, Rob H C and Boury, Himani and Albert, Mathieu and Chow, James and DaCosta, Ralph S and Hoffman, Michael and Keshavarz, Behrang and Kontos, Pia and Liu, Jenny and McAndrews, Mary Pat and Protze, Stephanie},
  title = {DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey},
  journal = {PLOS ONE},
  year = {2023},
  doi = {10.1371/journal.pone.0270616}
}
Kostenlos starten →