EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Metrics and peer review agreement at the institutional level

V. A. Traag, M. Malgarini, S. Sarlo

Quantitative Science Studies · 11.04.2025 · 6 · 328-350

DOI: 10.1162/qss_a_00352 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Quantitative Science Studies, online erschienen am 11.04.2025. Verfasst von V. A. Traag und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent decades, many countries have started funding academic institutions based on the evaluation of their scientific performance. Postpublication peer review is often used in this context. Bibliometric indicators have been suggested as an alternative to peer review. A recurrent question is whether peer review and metrics yield similar outcomes. In this paper, we study this question based on a sample of publications submitted to the national Italian research assessment exercise (2011–2014). In particular, we study the agreement between peer review and metrics at the institutional level, and compare this to the internal agreement of peer review. We base our analysis on a hierarchical Bayesian model using cross-validation. We find that the level of agreement is higher at the institutional level than at the publication level. Overall, the agreement between metrics and peer review is low, but on par with the similarly low internal agreement among two reviewers for certain fields of science. The low agreement between metrics and peer review is no reason for rejecting the use of metrics for some fields in the Italian national research assessment exercise. Our results provide input to the broader discussion of research evaluation, in which other factors also play an important role.

Zitieren

APA

Traag, V. A., Malgarini, M., & Sarlo, S. (2025). Metrics and peer review agreement at the institutional level. Quantitative Science Studies, 6, 328-350. https://doi.org/10.1162/qss_a_00352

Vancouver

Traag VA, Malgarini M, Sarlo S. Metrics and peer review agreement at the institutional level. Quantitative Science Studies. 2025;6:328-350. doi:10.1162/qss_a_00352

BibTeX
@article{Traag2025,
  author = {Traag, V. A. and Malgarini, M. and Sarlo, S.},
  title = {Metrics and peer review agreement at the institutional level},
  journal = {Quantitative Science Studies},
  year = {2025},
  volume = {6},
  pages = {328--350},
  doi = {10.1162/qss_a_00352}
}
Kostenlos starten →