Data and code are essential for ensuring the credibility of scientific results and facilitating reproducibility, areas in which journal sharing policies play a crucial role. Edward Ivimey-Cook from the School of Biodiversity, One Health & Veterinary Medicine, University of Glasgow (now a Lecturer at the University of East Anglia) tells us more about a new study that reviewed the clarity, strictness, and timing of data- and code-sharing policies across 275 journals in ecology and evolution, and assessed author compliance at two journals, Ecology Letters and Proceedings B.
About the study
The availability of data and code is crucial to ensuring the reproducibility and credibility of published research. Recently, more and more journals have begun to require or encourage authors to share data and code via specific sharing policies, typically upon article acceptance. Even though the number of journals with these policies has increased in the last few years, overall compliance with these policies is alarmingly low. Ultimately, low rates of data and code sharing have contributed to poor reproducibility across many research fields, and there is a lot of room for improvement.
An aspect that has often been neglected in meta-research studies is exactly when data or code sharing is required. Most research has focused on post-publication data- and code-sharing. This is surprising given the numerous benefits of providing data and code with first submission. Benefits like providing a deeper analytical insight for editors and reviewers, promoting reproducible workflows from the beginning of the project, and allowing the review of data and code through dedicated data editors - a role that's currently only present in seven journals in ecology and evolution, including Proceedings B.
The main goal of the work was to critically assess the timing (when does data and code need to be shared?), strictness (is the policy mandatory, encouraged, etc.?), and clarity (how easy is it for authors to understand what’s asked by the policy?) of policies for 275 ecology and evolutionary biology journals, the largest assessment to date. We also obtained data from hundreds of initial submissions from two journals, Ecology Letters and Proceedings B, to assess whether sharing policies are indeed associated with increased sharing. We found that less than half of journals publishing ecological and evolutionary research possess some form of mandated data-or code-sharing policy, and even fewer require authors to share this on first submission (i.e., during peer review). On the positive side, the number of journals mandating or encouraging sharing has increased since past surveys, albeit slowly. The analysis of data from initial submissions from Ecology Letters and Proceedings B showed that journals have a strong role in increasing data and code availability through these policies.
Visual depiction of the 7 points of advice for journals integrated in the publication process. Suggestions 1-4 involve the journal and/or publisher, suggestions 5-6 occur during peer review and article acceptance, and suggestion 7 occurs after publication to monitor the success of the process. Icons modified from Flaticon: Table-grid by Dave Gandy, Coding by Major Icons, regulation by IwitoStudio, people by Muhazdinata, copy by torskaya, magnifier by Creative Stall Premium. Figure by Romy Zeiss and Kevin Bairos-Novak.
How the idea for the paper came about
The idea of this paper came from a discussion between Alfredo, Nick, and I, about how previous work on journal data- and code-sharing policies had only focused on the clarity or strictness of policy, whereas the timing of when data and code are expected to be shared had never been investigated. The reason for focusing on timing was that there are multiple benefits for journals to request data and code at first submission - not least because it gives reviewers the opportunity to review them, but also because it enables data editors, if present in the journal, to assess data and code quality. We therefore decided to assess this feature of journals’ sharing policies and compare data- and code-sharing policies with previous assessments (in 2021 for data (last updated in 2023) and 2020 for code). At the same time, we wanted to gauge how compliant authors were to mandated sharing policies, but to do this, we needed submission data - that was when we specifically decided to contact Ecology Letters and Proceedings B for data on initial submissions. We then also decided that we could formally test whether the introduction of mandated policies increased sharing of both data and code. We then had the perfect opportunity to work on this with other researchers across ecology and evolutionary biology at the SORTEE 2023 conference in the form of a hackathon (a group event where there is a clear goal that everyone is working towards) that was open to anyone interested in the topic. After this main period of data collection but prior to any analysis, we pre-registered the study https://doi.org/10.17605/OSF.IO/ZXURH. For those unfamiliar with the term, pre-registration is the act of creating a detailed description of your research plan (i.e., the analyses you wish to perform) and storing it in a time-stamped and uneditable repository. This guards against Questionable Research Practices such a p-hacking (the manipulation of data to obtain significance) and Harking (hypothesising after results known) which can erode the credibility of research.
About the authors
Ed Ivimey-Cook - my research focuses on ageing and parental effects, in particular the long-lasting effects that parents, typically through their age and diet, can have on the traits of their offspring. In my spare time, I’m also passionate about open and reproducible science and am the acting president of SORTEE (The Society for Open, Reliable, and Transparent Ecology and Evolutionary biology).
Alfredo Sánchez-Tójar is an evolutionary ecologist with a soft spot for birds. He is committed to open, reliable, and transparent science and serves on the Board of Directors of SORTEE. His work combines evidence synthesis with meta-research to advance research practices and understanding in ecology, evolution, and beyond.
Nicholas Moran is an ecologist, variously working in the areas of invasion ecology and biosecurity, behavioural ecology, marine and freshwater science, and evidence synthesis and meta-science. Nicholas promotes the adoption of open science practices to improve the quality of ecological research and to support evidence-based environmental policy and management.
What was your experience like publishing in Proceedings B?
I (Ed) have always had a really great experience publishing in Proceedings B. In addition, and in particular with this submission, I have always valued the data editor stage at Proceedings B. It’s always incredibly constructive and useful to get someone else’s eyes on the data and code.
Proceedings B is looking to publish more high-quality biological science practices papers. If you have an idea for a paper or have one ready for submission, please see our website for more information on how to submit.
