Recommendations

Area for action: enhance access to essential AI infrastuctures and tools

Recommendation 1

Governments, research funders and AI developers should improve access to essential AI infrastructures

Access to computing resources has been critical for major scientific breakthroughs, such as protein folding with AlphaFold. Despite this, compute power and data infrastructures for AI research are not equally accessible or distributed across research communities (footnote 11).

Scientists from diverse disciplines require access to infrastructure to adopt more complex AI techniques, process higher volume and types of data, and ensure quality in AI-based research.

Proposals to improve access have included institutions sponsoring access to supercomputing (footnote 12) and the establishment of regional hubs – akin to a CERN for AI (footnote 13). Wider access can extend the benefits of AI to a greater number of disciplines, improve the competitiveness of non-industry researchers, and contribute towards more rigorous science by enabling reproducibility at scale.

Expanding access to computing must also be informed by environmentally sustainable computational science (ESCS) best practices, including the measurement and reporting of environmental impacts (footnote 14).

Actions to enhance access to AI infrastructures and tools may include:

1. Funders, industry partners, and research institutions with computing facilities actively sharing essential AI infrastructures such as high-performance computing power and data resources.
2. Relevant stakeholders (eg government agencies, research institutions, industry, and international organisations) ensuring access to high-quality datasets and interoperable data infrastructures across sectors and regions. This could involve advancing access to sensitive data through privacy enhancing technologies and trusted research environments (footnote 15).
3. Research funders supporting strategies to monitor and mitigate the environmental impact associated with increased computational demands and advancing the principle of energy proportionality in AI applications (footnote 16).

Area of action: enhance access to essential AI infrastuctures and tools

Recommendation 2

Funders and AI developers should prioritise accessibility and usability of AI tools developed for scientific research

Access to AI does not guarantee its meaningful and responsible use. Complex and high-performance AI tools and methods can be challenging for researchers from non-AI backgrounds to adopt and utilise effectively (footnote 17). Similarly, new skills are needed across the AI lifecycle, such as data scientists who understand the importance of metadata and data curation, or engineers who are familiar with GPU programming for image-based processing.

Taking steps to improve the usability of AI-based tools (eg software applications, libraries, APIs, or general AI systems) should therefore involve a combination of mechanisms that make AI understandable for non-AI experts and build their capacity to use AI responsibly. For example, training should ensure that every scientist is able to recognise when they require specialised data or programming expertise in their teams, or when the use of complex and opaque AI techniques could undermine the integrity and quality of results.

Improving usability can also enhance the role of non-AI scientists as co-designers (footnote 18) – as opposed to passive users – who can ensure AI tools meet the needs of the scientific community. Creating conditions for co-design requires bridging disciplinary siloes between AI and domain experts through the development of shared languages, modes of working, and tools.

Actions to enhance the usability of AI tools may include:

1. Research institutions and training centres establishing AI literacy curriculums across scientific fields to build researchers’ capacity to understand the opportunities, limitations, and adequacy of AI-based tools within their fields and research contexts.
2. Research institutions and training centres establishing comprehensive data literacy curriculums tailored to the specific needs of AI applications in scientific research. This involves building capacity for data management, curation, and stewardship, as well as implementation of data principles such as FAIR (Findable, Accessible, Interoperable, and Reusable) and CARE (Collective benefit, Authority to control, Responsibility, and Ethics) (footnote 19).
3. Research funders and AI developers investing in strategies that improve understanding and usability of AI for non-AI experts, with a focus on complex and opaque models (footnote 20). This can include further research on domain-specific explainable AI (XAI) or accessible AI tools that enhance access in resource-constrained research environments (footnote 21).
4. Research institutions, research funders, and scientific journals implementing mechanisms to facilitate knowledge translation across domains and meaningful collaboration across disciplines. This requires a combination of cross-discipline training, mentorship, publication outlets and funding (eg through bodies such as the UKRI’s Cross-Council Remit Agreement that governs interdisciplinary research proposals) (footnote 22).

Area of action: build trust in the integrity and quality of AI-based scientific outputs

Recommendation 3

Research funders and scientific communities should ensure that AI-based research meets open science principles and practices to facilitate AI’s benefits in science.

A growing body of irreproducible AI and machine learning (ML)-based studies are raising concerns regarding the soundness of AI-based discoveries (footnote 23, footnote 24). However, scientists are facing challenges to improve the reproducibility of their AI-based work. These include insufficient documentation released around methods, code, data, or computational environments (footnote 25); limited access to computing to validate complex ML models (footnote 26); and limited rewards for the implementation of open science practices (footnote 27). This poses risks not only to science, but also to society, if the deployment of unreliable or untrustworthy AI-based outputs leads to harmful outcomes (footnote 28).

To address these challenges, AI in science can benefit from following open science principles and practices. For example, the UNESCO Recommendation on Open Science (footnote 29) offers relevant guidelines to improve scientific rigour, while noting that there is not a one-size-fits-all approach to practising openness across sectors and regions. This aligns well with the growing tendency towards adopting ‘gradual’ open models that pair the open release of models and data with the implementation of detailed guidance and guardrails to credible risks (footnote 30)

Open science principles can also contribute towards more equitable access to the benefits of AI and to building the capacity of a broader range of experts to contribute to its applications for science. This includes underrepresented and under-resourced scholars, data owners, or non-scientist publics.

Further work is needed to understand the interactions between open science and AI for science, as well as how to minimise safety and security risks stemming from the open release of models and data.

Actions to promote the adoption of open science in AI-based science may include:

1. Research funders and research institutions incentivising the adoption of open science principles and practices to improve reproducibility of AI-based research. For example, by allocating funds to open science and AI training, requesting the use of reproducibility checklists (footnote 31) and data sharing protocols as part of grant applications, or by supporting the development of community and field-specific reproducibility standards (eg TRIPOD-AI (footnote 32)).
2. Research institutions and journals rewarding and recognising open science practices in career progression opportunities. For example, by promoting the dissemination of failed results, accepting pre-registration and registered reports as outputs, or recognising the release of datasets and documentation as relevant publications for career progression.
3. Research funders, research institutions and industry actors incentivising international collaboration by investing in open science infrastructures, tools, and practices. For example, by investing in open repositories that enable the sharing of datasets, software versions, and workflows, or by supporting the development of context-aware documentation that enables the local adaptation of AI models across research environments. The latter may also contribute towards the inclusion of underrepresented research communities and scientists working in low-resource contexts.
4. Relevant policy makers considering ways of deterring the development of closed ecosystems for AI in science by, for example, mandating the responsible release of benchmarks, training data, and methodologies used in research led by industry.

Area for action: ensure safe and ethical use of AI in scientific research

Recommendation 4

Scientific communities should build the capacity to oversee AI systems used in science and ensure their ethical use for the public good

The application of AI across scientific domains requires careful consideration of potential risks and misuse cases. These can include the impact of data bias (footnote 33), data poisoning (footnote 34), the spread of scientific misinformation (footnote 35, footnote 36), and the malicious repurposing of AI models (footnote 37). In addition to this, the resource-intensive nature of AI (eg in terms of energy, data, and human labour) raises ethical questions regarding the extent to which AI used by scientists can inadvertently contribute to environmental and societal harms.

Ethical concerns are compounded by the uncertainty surrounding AI risks. As of late 2023, public debates regarding AI safety had not conclusively defined the role of scientists in monitoring and mitigating risks within their respective fields. Furthermore, varying levels of technical AI expertise among domain experts, and the lack of standardised methods for conducting ethics impact assessments, limit the ability of scientists to provide effective oversight (footnote 38). Other factors include the limited transparency of commercial models, the opaque nature of ML-systems, and how the misuse of open science practices could heighten safety and security risks (footnote 39, (footnote 40).

As AI is further integrated into science, AI assurance mechanisms (footnote 41) are needed to maintain public trust in AI and ensure responsible scientific advancement that benefits humanity. Collaboration between AI experts, domain experts and researchers from humanities and science, technology, engineering, the arts, and mathematics (STEAM) disciplines can improve scientists’ ability to oversee AI systems and anticipate harms (footnote 42).

Similarly, engaging with communities represented in or absent from AI training datasets, can improve the current understanding of possible risks and harms behind AI-based research projects.

Actions to support the ethical application of AI in science can include:

1. Research funders and institutions investing in work that operationalises and establishes domain-specific taxonomies (footnote 43) of AI risks in science, particularly sensitive fields (eg chemical and biological research).
2. Research funders, research institutions, industry actors, and relevant scientific communities embracing widely available ethical frameworks for AI, as reflected in the UNESCO Recommendation on the Ethics of Artificial Intelligence (footnote 44), or the OECD’s Ethical Guidelines for Artificial Intelligence (footnote 45), and implementing practices that blend open science with safeguards against potential risks.
3. Funders, research institutions and training centres providing AI ethics training and building the capacity of scientists to conduct foresight activities (eg horizon scanning), pre-deployment testing (eg red teaming), or ethical impact assessments of AI models to identify relevant risks and guardrails associated with their field.
4. Research funders, research institutions, and training centres supporting the development of interdisciplinary and participatory approaches to safety auditing, ensuring the involvement of AI and non-AI scientists, and affected communities in the evaluation of AI applications for scientific research.