Article ; Online: The effect of the scale of grant scoring on ranking accuracy [version 2; peer review
F1000Research, Vol
2 approved]
2023 Volume 11
Abstract: In this study we quantify the accuracy of scoring the quality of research grants using a finite set of distinct categories (1, 2, …., k), when the unobserved grant score is a continuous random variable comprising a true quality score and measurement ... ...
Abstract | In this study we quantify the accuracy of scoring the quality of research grants using a finite set of distinct categories (1, 2, …., k), when the unobserved grant score is a continuous random variable comprising a true quality score and measurement error, both normally distributed. We vary the number of categories, the number of assessors that score the same grant and a signal-to-noise ratio parameter. We show that the loss of information of scoring a small number of categories (k > 5) compared to scoring on a continuous scale is very small, so that increasing the number of scoring categories is unlikely to lead to an improvement in the outcomes of scoring systems. In addition, we model the effect of grant assessors scoring too close to the mean and show that this results in only a very small reduction in the accuracy of scoring. |
---|---|
Keywords | grant scoring ; multiple threshold model ; grant quality ; grant ranking ; eng ; Medicine ; R ; Science ; Q |
Subject code | 006 |
Language | English |
Publishing date | 2023-02-01T00:00:00Z |
Publisher | F1000 Research Ltd |
Document type | Article ; Online |
Database | BASE - Bielefeld Academic Search Engine (life sciences selection) |
Full text online
More links
Kategorien
Inter-library loan at ZB MED
Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.