Then and now – exploring diversity in peer review at the Royal Society

Peer review cannot be done by everyone. It can only be done by people who share certain levels of training and subject-expertise, and have a shared sense of what rigorous experimentation, observation and analysis should look like. That shared expertise and understanding is what should enable alert peer reviewers to reject shoddy experimental methods, flawed analysis and plans for perpetual motion machines.

But as we have increasingly come to realise, any group of people with shared characteristics may display unconscious bias against outsiders, whether that means women, ethnic minorities, or those with unusual methods. While peer review should exclude poor science, it should not exclude good research on the basis of the individual traits or institutional affiliation of the researchers, nor should it dismiss innovative approaches to old problems.

However, it seems socio-cultural and intellectual criteria have often been mixed together in the peer review process, and history can help us to understand why.

Then

Crystallographer Kathleen Lonsdale (third from left) at the Royal Society in 1957. Credit: The Royal Society.

In the 1830s, the Royal Society was one of the first organisations to use a system of refereeing for papers under consideration for publication in the Philosophical Transactions. Using referees soon became a distinguishing feature of the research journals published by learned societies, in contrast to those managed by a sole editor. Using referees was not only a way of getting appropriate subject expertise, but it also limited the risks of one man’s prejudice or bias by ensuring that several people were involved in the process.

For the Royal Society, restricting the pool of reviewers to their own Fellows made good sense intellectually: membership of the Society not only indicated their demonstrable commitment to the sciences, but their participation in the weekly schedule of Society activities helped to ensure that they shared the cultural and scientific values that would help them recognise publishable science. However, it also meant that all those involved in making editorial decisions in the 19th century were well-educated, affluent, white, British males, over the age of 40.

Additionally, because the Royal Society has historically operated a system of refereeing in which the referees are anonymous but the authors are named, those white British males knew exactly who they were evaluating. Certain surnames could conjure up stereotypes about, for example, Indian or Japanese researchers and there was never any doubt about which authors were female, thanks to the allegedly chivalrous insistence upon giving the first names (and/or gender specific titles, such as ‘Mrs’) of female authors.

The diversity of people involved in scientific research increased through the 20th century, but the demographic profile of the Royal Society only slightly changed. By the 1960s there was somewhat greater diversity in the socio-economic backgrounds of the Fellows, thanks to grammar schools and university maintenance grants, and the average age had increased. But, still, only 3% of Fellows were women, even though women had been eligible for the Fellowship since 1945. (We don’t have statistics on ethnic diversity, but the Royal Society in the 1960s and 1970s was undoubtedly still predominantly white.)

In 1969, new editorial guidelines allowed for non-Fellows to be invited to act as referees. There is no evidence to suggest that this decision derived from any ambition to increase diversity. In fact, to our surprise, my colleague Camilla Mørk Røstvik and I discovered that fewer women were asked to referee for the Society in the 1970s than had been the case in the 1950s!

There was, however, some earlier awareness that having a limited pool of referees with strongly shared values might have some disadvantages. In 1892, for instance, the physicist Lord Rayleigh rediscovered a paper on thermodynamics which had languished in the Society’s archive since being rejected by referees in 1845 who had failed to appreciate its originality. Indeed, one of the original referees dismissed it as ‘nothing but nonsense’.

The tendency of referees to prefer papers dealing with topics, methods or theories that were familiar to them – and to reject the unfamiliar, the innovative or the speculative – raised more concerns in the early 20th century. For instance, Proceedings A became known as a home to papers dealing with quantum theory, relativity and nuclear physics during that time; but since the most active physicists in the Fellowship supported modern physics, they tended to dismiss papers that used classical approaches. By the 1930s, the worry that referees would miss innovation and novelty through their preference for the familiar led to explicit guidance about what should, or should not, merit rejection:

“A paper should not be recommended for rejection merely because a referee disagrees with opinions, theories or conclusion put forward by the author, unless in the course of the exposition unmistakeable logical fallacy or experimental error is made evident.” (Advice to referees, draft in Royal Society archive, Council Minutes vol. 15, 4 November 1937.)

Now

Biology Letters celebrating its 10th birthday in 2015.

It was only in 2015 that the Royal Society actively began trying to increase the diversity of people involved in its activities, with the creation of a Diversity Committee and the appointment of a Diversity Officer.

The results have been striking. In 1985, only 3% of Fellows were women; only 4% of papers submitted had one or more female authors; and just one paper was refereed by a woman. The latest figures for 2017 show that, although only 8% of Fellows are female, 33% of Royal Society authors were female, as were 30% of reviewers.

In addition, although the Royal Society’s procedures for electing Fellows may still be grappling with tradition its journals are attracting female authors, and its Editors (all bar one of whom are male) are managing to recruit female reviewers in almost the same proportion as authors – and certainly are doing better than Nature, where only 16% of reviewers are female.

So, while the de-coupling of refereeing from the Fellowship of the Society made no positive difference to women’s involvement in the 1970s, in the early 21st century, the efforts the Society has made to gather diversity statistics and raise awareness have made a real difference.

It’s why we have to keep on talking about diversity.