Peer review uncertainty at the institutional level

Abstract

A common element of all performance based research funding systems is the need to evaluate research. A recurrent question in this context is whether peer review and metrics tend to yield similar outcomes, or whether they differ substantially. We here study peer review uncertainty at the institutional level. We rely on data collected by ANVUR, the agency tasked with implementing the Italian research assessment exercise called VQR. We find that peer review agreement is generally higher at the institutional level than at the publication level. Similarly, correlations between peer review and metrics also tend to be higher at the institutional level. Finally, we find that the correlations between especially journal metrics and peer review are on par with correlations among two peer reviewers. Our results support the possibility of using metrics in combination with peer review for evaluation purposes.