Dr Rebecca Sear tells us her opinion on what makes a good or bad review, and how new reviewers can succeed in this process.
Following on from her previous post on ‘Tips for good practice in peer review’, Dr Rebecca Sear, Head of the Department of Population Health at the London School of Hygiene & Tropical Medicine and Editorial Board member of Philosophical Transactions of the Royal Society B, has given us some further thoughts about what makes a good or bad review, and how new reviewers can succeed in this process.
You’ve accepted your first peer review request, thoroughly read the paper, and are now sitting in front of a blank computer screen wondering what your peer review should look like. Here are my tips.
What does a good peer review look like?
1. Start with a (very) brief summary of the paper. This is a useful exercise for both reviewers and authors. If you struggle to summarise what the paper is about, that suggests the authors need to improve the clarity of their writing. It also lets the authors know what a reader took from their paper – which may not be what they intended!
2. Next, give the Editor an overview of what you thought of the paper. You will typically have to provide a recommendation (e.g. accept, revise or reject), but in the review itself you should give a summary of your reasons for this recommendation. Some examples:
- ‘the data appear appropriate for testing the authors’ hypothesis but I have some concerns about the methods. If these can be fixed, then this should become a useful contribution to the literature’.
- ‘the authors’ have a clear research question and use appropriate methods, but their data are not suitable to provide an answer to their research question. Without additional data collection, this paper is not appropriate for publication’.
3. The rest of your review should provide detailed comments about the manuscript. It is most helpful to Editors and authors if this section is structured in some way. Many reviewers start with the major problems first, then list more minor comments afterwards. Major comments would be those which need to be addressed before the paper is publishable and/or which will take substantial work to resolve – such as concerns with the methodology or the authors’ interpretation of results. Minor comments could be recommendations for revisions that are not necessarily essential to make the paper publishable – for example, suggestions for additional literature to include, or cosmetic changes.
4. Remember that you have two audiences: the Editor and the authors. Authors need to know what was good about the paper and where improvements could be made. The Editor needs to know if you think the manuscript is a publishable piece of work. Bear in mind that different journals have different criteria for what makes a paper publishable – this information should be accessible on the journal webpage, or you might have been sent guidance to help with this when you accepted the invitation to review.
5. Your review should be clear, constructive and consistent.
- Clarity is important because authors will not be able to respond to your concerns if they don’t fully understand what they are.
- Reviews are most helpful if they don’t just criticise, but also make constructive suggestions for how concerns may be resolved.
- Your overall recommendation should be consistent with your comments. There is likely to be an opportunity to provide confidential comments to the Editor to provide further context or justification for your recommendation, but don’t include comments here that are completely different from the main messages of your review. The Editor needs to be able to justify their final decision to the authors using the reviewer comments as part of their evidence.
6. Don’t be afraid to highlight good things about the paper – a good review does not just criticise but also highlights what the authors have done well.
7. Your review should always be polite; it is unprofessional to use derogatory language or take a harsh or sarcastic tone (and remember that even if reviewer names are blinded to authors, the Editor knows who you are…). Write the review in a tone you would be happy to receive.
What does a bad review look like?
1. One which uses insulting or unprofessional language.
2. An unstructured stream of consciousness, which lists many concerns but without any indication of which are the most serious.
3. One in which the comments are too brief for the Editor to understand what you liked or didn’t like about the paper.
4. One which doesn’t justify its recommendation, or makes a recommendation which is not reflected in the comments. The Editor needs to know why you came to the recommendation you did.
5. One which insists on unnecessary revisions – maybe the paper would be improved by another seven experiments, but that’s not relevant to the question of whether the paper you have in front of you is publishable. You should feel free to make suggestions about additional work that you think might improve the paper, but don’t insist on this unless the paper in front of you isn’t publishable without it.
6. One which makes unreasonable demands. Think about how the authors are going to respond to your concerns, and only recommend changes which are feasible.
Peer reviewing is an opportunity to improve the quality of published research, and is very valuable to the research process when it works well. I hope these tips are helpful for making the process as efficient (and painless!) as possible.
Visit our website for more information on reviewing for the Royal Society’s journals.
