Tips for good practice in peer review

Quality in peer review

We spoke to Dr Rebecca Sear, Head of the Department of Population Health at the London School of Hygiene & Tropical Medicine and member of the Editorial Board of our journal Philosophical Transactions of the Royal Society B, about quality in peer review. Rebecca has developed a training course on ‘Good practice in peer review,’ which she presents to students at her institution. 

Over the 20 years that I’ve been engaging in the peer review process, as author, reviewer and (more recently) editor, it’s become clear that the peer review system could be improved. While I’ve seen many examples where peer review worked extremely well to improve the quality of published research, I’ve also come across many cases where the system fails to do that job properly. The most high-profile failures of peer review happen when insufficiently rigorous review lets slip poor quality research into the literature. But the review system also fails because of overly harsh or inappropriate (for example, recommending rejection on the basis of null results) review. At worst, this distorts science by keeping valuable research out of the literature entirely. It also creates inefficiencies in the system when publishable research has to be submitted to multiple journals before publication, burdening several reviewers and editors with the costs of evaluating the same research. A further problem is that the anonymity typically given to peer reviewers can result in unprofessional behaviour being unleashed on authors.

Despite its problem, the clear benefits to receiving detailed, constructive feedback from your peers means that I’m not yet ready to decide that peer review is dead, so I’ve written a training course for early career researchers on how to peer review constructively. One reason for poor practice in peer review may be that, until recently – and as with so many other roles academics have to perform – there has been little training for peer review.

Some of this training course involves simply promoting good research practices. For example: don’t recommend rejection just because a study finds null results. Part of promoting good research practices also involves facilitating interdisciplinary research and communication, certainly for those of us working in the human sciences. My research spans the biological, social and health sciences and it’s clear from my experiences with journals from multiple disciplines that the disciplinary silos academia has constructed for itself further hinder the progress of research. Different disciplines have different social norms when it comes to the process of peer review: social science journals could learn greater efficiency from the biological science norm of typically inviting only 2 reviewers for each paper (4 or more is REALLY not necessary). Of more concern for the quality of the published record, different disciplines also have different norms when deciding what is ‘good’ research: the biological sciences could learn from some of the social sciences by toning down their desire for ‘novel’ findings.

What this means when recommending good practice in peer review is that reviewers should focus on aspects of the manuscript that can be objectively assessed (data and methods) and less on more subjective judgements (on novelty, on how the paper is structured or framed), since the latter may differ according to each discipline’s social norms. Reviewers should also be aware that different disciplines do have different norms, and learn to embrace this diversity by not insisting research replicates the norms of their own discipline.

My top tips for peer review

1. Respond promptly to a review request, whether you are accepting or declining; if declining, use the opportunity to recommend other early career researchers (ECRs) (who may not be on the editor’s radar) as alternative reviewers
2. Think about your audiences: your comments need to both aid the Editor in making a decision, and also provide constructive feedback to the authors
3. Focus first on the methodology and data. If the data, methodology and analysis is suitable and sound, then problems with interpretation and presentation can usually be easily fixed
4. Evaluate the manuscript you have in front of you; do not be tempted to insist the authors write the paper that you would have written instead
5. Ask yourself whether all of the information is available to replicate the authors’ study and interpret their results. Many journals now ask for all data to be made available – check this if applicable
6. Reviewers can add considerable value by ensuring that the paper is clearly and appropriately presented. For example: is the manuscript clearly written, does the abstract adequately summarise the study, are figures are a fair and clear representation of the results?
7. It is not the reviewer’s role to ‘edit’ the language or presentation, but you should make suggestions to the author to improve readability, and flag to the editor if the quality of the language makes it too difficult to understand the scientific content
8. Don’t just focus on the paper’s contents – flag to the Editor any concerns regarding ethics and research integrity, data availability, transparency etc.
9. Let the editor know if there are aspects that you don’t feel competent in assessing
10. Read your review carefully before submitting. Make sure that it is clear and constructive, doesn’t use jargon or potentially confusing terminology, and is polite and professional. Ensure that attachments don’t include your personal details unless you are willing to give your name to the authors

Finally, remember the Golden Rule (in peer review as in all things): review others as you want to be reviewed yourself.

Access the full slide set for Rebecca’s training course on ‘Good practice in peer review’, which also includes links to useful online resources. You might also be interested in Rebecca's follow-up blog on 'What makes a good or a bad peer review? Tips for excelling at reviewing'.

Find more information on reviewing for the Royal Society’s journals.