A look at the Society’s efforts to ensure a diverse reviewer base.
This year’s peer review week aims to highlight the theme of diversity in peer review. We have already looked at how diversity in peer review has evolved over the Royal Society’s history. Now we look at the Society’s current efforts to ensure a diverse reviewer base.
Diversity in peer review is essential to delivering the very best published science. Review by peers has been the basis of scientific publishing since its inception – at first informally then latterly using dedicated reviewers.
When assessing a piece of research, journals strive to identify the best people with the knowledge to properly review it. If the journal limits the diversity of this pool it is not using the broadest range of expertise available in the international scientific community. However, selecting from a much larger, diverse and international pool of researchers increases the chances of identifying (and appointing) the very best reviewers.
“Research needs diversity and diversity means diverse people, living diverse lives with diverse approaches and diverse experiences”
Professor Ottoline Leyser
Here we look at some of the challenges in diversifying peer review and how we might overcome them.
Tackling unconscious bias
Unconscious bias is when we make judgments or decisions on the basis of our prior experience, our own personal deep-seated thought patterns, assumptions or interpretations, and we are not aware that we are doing it. (The Royal Society, Unconscious bias briefing)
In 2015 the Society produced a briefing document (PDF) and animation for all committees explaining what unconscious bias is and what can be done about it. Prior to the start of each journal Editorial Board meeting, we have made a point of briefing members about unconscious bias and urged them to take it into account when making decisions. This message has been extended to all reviewer invitations.
Improving the diversity of our Editorial Boards
Editorial Board members have a central role in suggesting reviewers. If the Editorial Board is not very diverse, it is likely to limit the diversity of recommended reviewers. In recent years, we have been increasing Editorial Board diversity through broader recruitment. A major focus has been gender diversity. According to the most recent audit 39% of Board members are female, although this varies by journal and scientific discipline. This is encouraging: figures reported across the scientific population as a whole show four out of five SET (science engineering technology) professors are male. We continue to recruit a more diverse range of scientists to Editorial Boards.
International diversity of reviewers
A study of life scientists in 2015 shows that the proportion of researchers acting as authors and reviewers varies considerably across the world. For instance, researchers in China are publishing twice as many papers as they are reviewing. Conversely, reviewers in North America and Europe are overburdened – our data reflects this with over three quarters of our reviewers based in these regions.
More generally, there is a large number of qualified scientists from outside North America and Europe (particularly early in their careers) who have never been invited to review. Researchers from outside these regions are an important untapped resource for journal publishers. Initiatives such as Publons Academy which train early career researchers across the world need to be encouraged.
Publication of peer review reports
In the past we have highlighted the benefits of publishing peer review reports. By default these are published anonymously. However, we do encourage reviewers to sign their reports so they can get recognition for the work. This is win-win for reviewers: their identity is protected if they have concerns about providing critical comments to more senior colleagues; alternatively, if they sign they can get credit for the specific report and include this in their Publons profile or ORCID record. Recognition of peer review work is becoming increasingly important in grant, tenure and career development.
We recognize the power of publishing peer review information and several of our journals have signed up to doing this through the ASAPBio open letter.
Looking forward
Machine learning is starting to be employed to match reviewers with scientific papers. Provided the algorithms do not introduce unnecessary biases, such systems can recommend a list of reviewers based on their subject expertise alone. Editors (who are conscious of their unconscious bias) can then select from a much wider pool of reviewers. The Society is piloting such a reviewer recommendation system called Publons Reviewer Connect.
Clearly there is still much work to do in improving diversity in peer review. The good news is that it is high on the agenda of many organizations, as highlighted by this year’s Peer Review Week.
We encourage all suitably qualified scientists to actively participate in peer review, whoever you are and wherever you come from. Find out more about the benefits of reviewing for the Royal Society.
