How do you feel about open peer review?

Opening up the review process

Royal Society Open Science operates the following model. At all points in the peer review process we encourage referees to sign their reports, thereby disclosing their name to the author. Where authors agree, we make the editorial process transparent by publishing the referee reports, the decision letter and associated author responses, alongside the actual manuscript. This allows readers to better assess the published paper and provide post-publication comments, which are also published. If the referees have opted to disclose their names, then these are included alongside the published reports. An outline of the possible scenarios is listed below:

• Author agrees to open peer review – referee agrees to open peer-review
  Signed referee report published
• Author does not agree to open peer review – referee agrees to open peer-review
  Referee name only disclosed to author, referee report is not published
• Author agrees to open peer review – referee does not agree to open peer-review
  Referee name not disclosed to author, anonymized referee report published
• Author does not agree to open peer review – referee does not agree to open peer-review
  Referee name not disclosed to author, referee report is not published

Here’s an example of the first scenario, a paper from August this year on improving body mass estimates in mammals.

With Peer Review Week upon us, we decided to ask the editorial board of Royal Society Open Science how they feel about open peer review. Here’s what they said:

• Transparency is good
• This is a great idea, it is likely to make peer review more thoughtful and balanced
• Neutral
• Mixed feelings. On the one hand, I like the transparency and it tends to discourage biased or unfair reviews. On the other hand, it may put some referees off and make them reluctant to be as honest in their criticisms as they might otherwise be
• I do not like open peer reviews – there are a lot of vicious people out there

A mixed bag

Earlier this year, nine months after the launch of Royal Society Open Science, we held a series of lively debates here at the Royal Society on the Future of Scholarly Scientific Communication (FSSC), including sessions on peer review. You can read arguments for and against peer review, but we also discussed its future development, evolution and alternative practices.

The following is an edited extract from the full FSSC report.

Can we reform peer review?

A lot of discussion cantered on the need to separate dissemination from verification, and that peer review should be done directly by the scientific community rather than through publishers who impose high costs on the process. One way of achieving this is by the use of pre-print repositories. Physicists pioneered this with ArXiv in the 90s and findings are often, though not invariably, published in a journal thereafter – we now welcome submissions directly from ArXiv. The advantages of pre-print servers are:

• Speed; articles are made public immediately
• Open and collaborative feedback
• Opportunity to improve a paper before submission to a journal
• Greater chance of errors being picked up and corrected due to greater exposure pre-publication

A system of universal, free-to-view and well-regulated preprint repositories with good trackback mechanisms for comments was seen by many to be the most desirable model. Of course, there are improvements to be made to a pre-print system too such as ways to provide recognition for reviewers, improving cross search functionalities and ensuring compatibility with existing servers.

The principle of peer review is necessary and valuable

Given how well such systems work in certain communities already – such as physics – the need for journals at all was questioned. Nevertheless, suggestions for reforming peer review based on the existing publisher-journal model were helpfully debated and included:

• Open peer review as standard with signing of reports optional
• Use of objective peer review, i.e. reviewing for correctness rather than impact
• Collaboration between all referees to produce a single review
• Portable peer review with authors owning their reports and able to take them from journal-to-journal, following a rejection for example
• Universally recognized systems for crediting peer reviewers

The overall view was that the principle of peer review was necessary and valuable, but should be organized in a different way. In general, it was felt that opportunities offered by new technologies and the web had not yet been fully exploited, and there was a role for learned societies and funders to encourage innovation and drive the necessary changes. We believe that the option of open peer review on journals such as Royal Society Open Science is a step in the right direction.