Metrics and peer review agreement at the institutional level
Quantitative Science Studies · 11.04.2025 · 6 · 328-350
DOI: 10.1162/qss_a_00352 ↗Kurz gefasst
Fachartikel in der Zeitschrift Quantitative Science Studies, online erschienen am 11.04.2025. Verfasst von V. A. Traag und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In recent decades, many countries have started funding academic institutions based on the evaluation of their scientific performance. Postpublication peer review is often used in this context. Bibliometric indicators have been suggested as an alternative to peer review. A recurrent question is whether peer review and metrics yield similar outcomes. In this paper, we study this question based on a sample of publications submitted to the national Italian research assessment exercise (2011–2014). In particular, we study the agreement between peer review and metrics at the institutional level, and compare this to the internal agreement of peer review. We base our analysis on a hierarchical Bayesian model using cross-validation. We find that the level of agreement is higher at the institutional level than at the publication level. Overall, the agreement between metrics and peer review is low, but on par with the similarly low internal agreement among two reviewers for certain fields of science. The low agreement between metrics and peer review is no reason for rejecting the use of metrics for some fields in the Italian national research assessment exercise. Our results provide input to the broader discussion of research evaluation, in which other factors also play an important role.
Zitieren
APA
Traag, V. A., Malgarini, M., & Sarlo, S. (2025). Metrics and peer review agreement at the institutional level. Quantitative Science Studies, 6, 328-350. https://doi.org/10.1162/qss_a_00352
Vancouver
Traag VA, Malgarini M, Sarlo S. Metrics and peer review agreement at the institutional level. Quantitative Science Studies. 2025;6:328-350. doi:10.1162/qss_a_00352
BibTeX
@article{Traag2025,
author = {Traag, V. A. and Malgarini, M. and Sarlo, S.},
title = {Metrics and peer review agreement at the institutional level},
journal = {Quantitative Science Studies},
year = {2025},
volume = {6},
pages = {328--350},
doi = {10.1162/qss_a_00352}
}
EUVIMED