During the time of a pandemic, there is an inevitable clamour for scientific research.
As scientific publishers, we know that our peer review and processes are vital to the publication of high-quality research. We know the importance of securing the most suitable reviewers, the significance of checklists around data, ethics and plagiarism, and the ideals of open peer review and transparency. But once an article has been published and set loose into the cybersphere, how can the reader of that research be sure that it is trustworthy? This post is here to help.
During the time of a pandemic, there is an inevitable clamour for scientific research - from scientists seeking the latest findings, officials and economists looking to create policy and plans, journalists wishing to inform the public and that same public looking to inform themselves. When we add the ingredients of preprints and press releases, together with an increased volume of research at a time when speed is of the essence, it is hardly surprising that it is sometimes hard to see the wood for the trees.
Here are our top tips for spotting the trustworthy research:
Study design
- Look for evidence that the study was designed to answer an initial hypothesis or question, rather than framing a question around the results.
- Consider whether it is really big enough to justify the conclusions.
- Check that it has been replicated enough times by the research group to get reliable results.
- Ensure that there are suitable controls in place.
- Is the experimental design suitable for any statistical tests that will be used to analyse the data?
- Study designs that are pre-registered help verify whether the authors performed the study as intended and described, rather than back-forming any protocols and hypotheses.
Publication
- Has the research been through a rigorous peer review process or is it still at preprint stage.
- Use publicly available lists of reputable journals to check the publication in which the research appears. How long has the publisher existed? What are their ethics (are they enrolled in COPE)?
- Use the ‘submitted’, ‘accepted’ and ‘published’ dates listed on the paper (if available) to see the history of the paper. An extremely fast turn-around may be an indication of a lack of thorough review (although this is not always the case).
Supporting information
- Many journals require authors to make their data available alongside the paper. You can use this data to do your own checks on the research, whilst at the same time being reassured that the data is there for other experts to check too.
- Some journals, including several of our own, publish peer review reports, decision letters and author responses alongside the paper. These will highlight any concerns that were raised during the review process.
- Look at the metrics and usage, both of which can usually be found at article level. Platforms like Altmetric track a range of sources to capture and collate activity and discussions around the research.
“The entire [scientific research] process is based on honesty, openness and transparency, in which the evidence is published for all to see and argue about. It is no coincidence that scientists are highly trusted" - Venki Ramakrishnan, President of the Royal Society. Read more in Venki’s Following the science blog, in which he addresses trust in science during the COVID-19 pandemic.
