Phil Hurst discusses the decision to collect gender data for Royal Society journals with the aim to identify and respond to potential biases in the peer review process.
In response to the theme of Peer Review Week 2021; Identity in Peer Review, Phil Hurst writes about how we decided to collect gender data to identify and respond to potential biases in the peer review process.
Since 2017 we’ve collected generic data via surveys on the gender of authors who submit papers to our journals. What prompted us to take this further so we could assess biases during peer review? In two words: chemistry and COVID-19.
Chemistry
Back in 2019, the Royal Society of Chemistry published the report Is publishing in the chemical sciences gender biased?. The answer is ‘yes’ and you can explore more details in the report.
They called on other scientific publishers to commit to action and report on their own activities. This led to us joining, along with many publishers, a commitment for action on inclusion and diversity in publishing.
How does COVID-19 fit in?
During the COVID-19 pandemic public health measures required people to stay home. Studies found that female scientists submitted substantially fewer manuscripts than male scientists. This was particularly pronounced for manuscripts about coronavirus. Such findings may be partly explained by the heavier teaching and caregiving responsibilities that women have.
As a high-profile member of COVID-19 rapid review initiative, we were asked several times if this submission pattern was reflected in the Society’s journals. We could not confirm as we didn’t have the specific data.
The problem with collecting gender data
As mentioned above, the Society has been collecting diversity data for several years via an annual survey. This includes editorial boards, authors, and reviewers. Surveys provide good data but have weaknesses due to response rate and who responds. It does not help with identifying biases during peer review.
An alternative approach is to analyse gender using algorithms which attempt to match a gender to first names. This has drawbacks including a significant proportion of names where gender cannot be determined.
These limitations led to the conclusion of members of the joint commitment; to collect gender data, with permission, via journal online peer review systems such as ScholarOne Manuscripts or Editorial Manager.
Collecting gender data during peer review
Our approach is quite simple: when an author submits a paper, they are prompted to describe their gender via a picklist (or select ‘prefer not to say’). We are transparent about who has access to the data and how it will be used and protected. Our approach mirrors that of the Royal Society of Chemistry and the Institute of Physics – the results are similar, with around 95% of submitting authors opting to describe their gender.
There is power in this simple approach – we can easily assess bias at each stage of the peer review process: desk reject, triage, reviewer recommendation and editor decision. Having this information is an important tool to address such biases.
This is just the start. Once we and other publishers have agreed a standardised approach, we will be able to combine anonymised gender data to help the scholarly publishing community as a whole to tackle gender bias in peer review. I encourage others to join us on this journey.
Find out about how to volunteer to review for the Royal Society’s journals, and the benefits that we offer our reviewers.
Image credit - Photo by Ryoji Iwata on Unsplash
