Member Research and Reports

Member Research and Reports

Johns Hopkins Researchers: Peer Review System for Awarding NIH Grants is Flawed

The mechanism used by the National Institutes of Health (NIH) to allocate government research funds to scientists whose grants receive its top scores works essentially no better than distributing those dollars at random, new research suggests.

The findings suggest that the expensive and time-consuming peer-review process is not necessarily funding the best science, and that awarding grants by lottery could actually result in equally good, if not better, results. A report on the research, published online Feb. 16 in the journal eLife, was written by Dr. Ferric Fang, at the University of Washington, Mr. Anthony Bowen,  at the Albert Einstein College of Medicine, and Dr. Arturo Casadevall, at the Johns Hopkins Bloomberg School of Public Health.

“The NIH claims that they are funding the best grants by the best scientists. While these data would argue that the NIH is funding a lot of very good science, they are also leaving a lot of very good science on the table,” says Dr. Casadevall, professor and chair of the W. Harry Feinstone department of molecular microbiology and immunology at Johns Hopkins. “The government can’t afford to fund every good grant proposal, but the problems with the current system make it worse than awarding grants through a lottery.”

Notes Dr. Fang, a professor of laboratory medicine and microbiology at the University of Washington: “We are not criticizing the peer reviewers. We are simply showing that there are limits to the ability of peer review to predict future productivity based on grant applications. This suggests that some of the resources and effort spent on ranking applications might be better spent elsewhere. While the average productivity of grants with better scores was somewhat higher, the differences were extremely small, raising questions as to whether the effort is worthwhile.”

Read more: