Royal Society Open Science has published its first Registered Report, a format of empirical article where a study proposal is reviewed before the research is undertaken.
This week Royal Society Open Science published its first Registered Report. Registered Reports are a type of research paper that have been developed by, amongst others, one of our very own Editors, Professor Chris Chambers. Having overseen the editing of The effects of exposure to objective coherence on perceived meaning in life: a preregistered direct replication of Heintzelman, Trent & King (2013), Chris talks a little about the article type and some of the experiences he has had editing the journal’s first.
What is a Registered Report and how is it different from a ‘regular’ research paper?
Registered Reports are a format of empirical article where a study proposal is reviewed before the research is undertaken. Pre-registered proposals that meet high scientific standards are then provisionally accepted before the outcomes are known, independently of the results.
Why are they an important innovation?
The aim of this initiative is to eliminate various forms of bias from the publication process. Most areas of science suffer from publication bias, where results that are positive and clear are easier to publish than those that are negative or inconclusive, even when the methodological quality of the studies is the same. By provisionally accepting articles before results exist, we abolish this bias.
In many fields, publication bias drives various forms of research bias, in which researchers (even unconsciously) selectively report results that are easier to publish, or even change their hypothesis in order to “predict” an unexpected finding. Registered Reports neutralise this bias by requiring authors to commit to their hypotheses and analysis plans before they commence their research.
The broader goal of Registered Reports is to reshape the incentive structures in science so that individual researchers are rewarded for simply doing the best and most transparent science they can, over and above their ability to craft good stories. Until we align the practices that are best for scientific advancement with the set of practices that are important for career success, many areas of science will experience low reproducibility.
What are the benefits for authors, journals and Editors in encouraging Registered Reports?
Authors benefit by having their papers provisionally accepted in advance of completing their research, eliminating the risk that the results may be insufficiently interesting, novel, or conclusive to warrant publication. The format also prevents publication being blocked by reviewers on the basis of the results differing from expectations or desires. On the other hand, intensive pre-study review can help solve problems in study designs before they occur, maximising the quality and impact of a study and overcoming a major source of stress for authors in the publication process. The format also provides a route for researchers to conduct replication studies and other novel, resource-intensive projects that may otherwise be too risky to attempt where successful publication is contingent on the results.
Editors (and reviewers) benefit by having the ability to contribute to study design before research is undertaken, and being able to observe the evolution of a study from conception to completion.
Finally, journals benefit by increasing the stock of research that is transparent and reproducible, and thus likely to make a meaningful, longer-term contribution to science and its applications.
In an earlier blog post, you commented on some of the challenges that face Editors in handling Registered Reports. As you’ve now had some ‘hands-on experience’ for Royal Society Open Science, have your thoughts about the challenges changed and, if so, why?
I think these challenges are omnipresent and universal to Registered Reports, at all the journals that have so far offered them (43 journals, to date). To satisfactorily edit a Registered Report requires nothing less than achieving Dorothy Bishop’s 'paragon' status as an editor. As editors we must adhere strictly and carefully to the journal policy, ensuring that papers are not accepted or rejected for the wrong reasons, and that study quality is maintained. My experience at Royal Society Open Science has, so far, been universally positive in this respect. One of the hallmarks of Registered Reports, here as at other journals, is that the review process tends to be more constructive and collaborative than for conventional articles.
Are there any fields for which Registered Reports may be a more ‘natural’ fit than others? What are these and why might this be the case?
Registered Reports are suitable for any area of science where there is bias – either bias by journals in deciding that some results are more publishable than others, or bias by researchers in selecting which aspects of a study to report. Some fields, including many life sciences, experience strong bias in both domains, due to the pressures on researchers to get “good results” together with a reliance on statistical methods that enable many paths to achieving them. The physical sciences also experience publication bias but are perhaps less susceptible to research bias – however, for Registered Reports to have value, all that is required is that some form of bias influences either the science or the publication process.
Not all types of science are equally amenable to Registered Reports. The format celebrates the deductive scientific method, so purely exploratory research or observations (i.e. with no prior hypothesis) are less applicable. Pure methods development is also unlikely to be suitable for a Registered Report. Of course, not all cases of exploratory research or methods development will be unsuitable, and editors are happy to consider proposals on a case-by-case basis.
Next week, we hear from the authors about their experience with the Registered Reports format.
Royal Society Open Science is committed to improving the reproducibility and transparency of published research. We encourage the submission and publication of pre-registered studies as Registered Reports. Interested authors considering submitting to the journal should visit the website to find out more about Registered Reports.
