Authors and reviewers can now provide feedback on their experience with Registered Reports at different journals.
Royal Society Open Science has been publishing Registered Reports since 2016. A new feedback website from a team at Cardiff University aims to support the growing Registered Reports community. Ben Meghreblian from the team tells us more:
Over the past decade there have been significant leaps forward in innovating scientific publishing, including in the area of Registered Reports. In this format, study proposals are peer reviewed before research has been conducted, and journals commit to publishing the final research based solely on the rationale and methodological rigour of the study proposal. By doing so, it seeks to address weaknesses with standard preregistration, guard against various forms of reporting bias and publication bias, and improve research quality. While early evidence suggests that Registered Reports aid reproducibility and research quality, uptake of the format by authors and journals remains limited. This may be due to perceptions and experiences regarding what the format offers, what it allows, and how it operates. Additionally, there may be hidden variation across journals in the quality of editorial handling.
Negative impressions of Registered Reports could be impacted by numerous experiences. Some researchers may view the format as overly constraining, believing that Registered Reports are only for confirmatory research that doesn’t allow flexibility in reporting exploratory or incidental findings. Or, an author who submits a Registered Report and has a bad experience at a journal may attribute the problem to the format rather than the journal. Alternatively, if a reviewer has been assigned to review a Registered Report, but has not fully understood the format or the scope of their review, then this may lead to both the reviewer and author with a confused image of what Registered Reports have to offer. With traditional peer review, community projects exist to gather information about researchers and authors’ experiences. However, with Registered Reports, no such resource exists.
Our work is seeking to address this. The broad goal of our latest project – funded by the Arnold Foundation – is to collect information regarding how well various aspects of the Registered Reports process are implemented across academic journals. We’re doing this through the Registered Reports Community Feedback website, which we launched at the end of 2022. This website collects anonymous feedback from authors and reviewers of Registered Reports manuscripts, asking a range of questions about their experiences of the peer review process. The website also presents dashboards, as shown below, which show aggregate quality and speed ratings of Registered Reports across a range of journals.
We hope the website will serve multiple functions. It may help authors choose where to submit their Registered Report manuscripts, and it may incentivise publishers to improve the Registered Reports process at their journals. Crucially, it will help journals and publishers understand what they're doing well, what they might do better, and how they can improve and innovate the review process for Registered Reports overall.
We also hope the website will contribute to new findings in the field of metascience. I am a member of a research team based at Cardiff University which focuses on the adoption and improvement of Registered Reports. This project looks at questions such as: How fast can Registered Reports peer review be? What’s the quality like? How do Registered Reports compare to traditional peer review? What are the key predictors of good or bad experiences around Registered Reports from an author perspective?
You can support this work by sharing your experiences of Registered Reports. If you've been an author or reviewer of a Registered Report manuscript at Stage 1 and/or Stage 2, you can find our survey here (takes 5-10mins).
If you edit for a journal that offers Registered Reports, you can follow the lead of Royal Society Open Science by pointing your authors and reviewers to our website.
Find out how to prepare and submit a Registered Report article to Royal Society Open Science, and read all published Registered Reports in our Reproducibility and transparency collection.
Image credits
Top image: Photo by Miguel Á. Padriñán https://www.pexels.com/photo/two-white-message-balloons-1111368/
Registered Reports Community Feedback logo - provided by Ben Meghreblian
Registered Reports Community Feedback dashboard screenshot - provided by Ben Meghreblian
