Choose Your Moments: Peer Review and Scientific Risk Taking / Richard T. Carson, Joshua S. Graff Zivin, Jeffrey G. Shrader.

Format:

Book

Author/Creator:

Carson, Richard T.

Contributor:

National Bureau of Economic Research.

Graff Zivin, Joshua S.

Shrader, Jeffrey G.

Series:

Working Paper Series (National Bureau of Economic Research) no. w31409.

NBER working paper series no. w31409

Language:

English

Physical Description:

1 online resource: illustrations (black and white);

Place of Publication:

Cambridge, Mass. National Bureau of Economic Research 2023.

Summary:

Science funding agencies such as the NIH, NSF, and their counterparts around the world are often criticized for being too conservative, funding incremental innovations over more radical but riskier projects. One explanation for their conservatism is the way the agencies use peer review of scientific proposals. Peer review is the cornerstone of research allocation decisions, but agencies typically base decisions on a simple average of peer review scores. More novel ideas are less likely to gain consistently high ratings across evaluators and are less likely to be funded. Using a discrete choice experiment conducted with a large sample of active biomedical researchers, we find that--in contrast to funding agencies--scientists systematically prefer to fund projects with more reviewer dissensus. Rather than purely focusing on the first moment of the distribution of reviewer scores, they also value the second moment. Further, scientists with the greatest domain expertise on a proposal are more enthusiastic about dissensus, and while appetite for dissensus shrinks as budgets become tighter, it does not disappear completely. Applying our estimates to prior studies mimicking NIH's review process shows that incorporating expert scientists' preferences for dissensus would change marginal funding decisions for ten percent of projects worth billions of dollars per year.

Notes:

Print version record

June 2023.