Uncertainty in science

Uncertainty in science

An article written by Professor Tim Palmer FRS.

Recognition of uncertainty is a characteristic of the scientific method and can be viewed from different aspects – mathematical, philosophical and statistical. Important decisions are made, in government and business, in the light of uncertain scientific advice, yet the methods by which different scientific disciplines assess and communicate uncertainty are rarely compared. A cross-fertilisation of ideas for how to represent and communicate uncertainty could have enormous benefits on our understanding of everything from economics to health issues or climate change.

Professor David Spiegelhalter FRS gives his thoughts on uncertainty in science. (4 mins, requires Flash Player).

Recent public debate about climate change has undoubtedly demonstrated that uncertainty in science needs to be more effectively explained.  Despite a growing number of suggestions that the science of climate change is becoming more uncertain, weather and climate scientists have, in fact, pioneered techniques to assess uncertainty in the evolution of complex nonlinear systems and our understanding is growing more confident.  However, we are not yet fully exploiting the inherent value of our knowledge of uncertainty in communicating with business, government, the media and the public.

    Audio
    Professor Dame Sally Davies on evidence-based policy-making
    (2 mins).
  • Listen here.

Projection of trajectory of Lorenz system in phase space.

Uncertainty is a complex and broadly defined phenomenon, with many possible categorisations and disciplinary approaches.   A simple but useful distinction is between uncertainty about what might happen in the sense of chance, randomness or essential unpredictability, and uncertainty about facts due to lack of knowledge or even ignorance. 

Randomness occurs naturally in games of chance or in the fundamental laws of physics as described in the axioms of quantum mechanics.  But even within classical physics, with its deterministic and precisely known laws, Ed Lorenz’s prototype model of chaos shows that for many nonlinear systems long-term prediction is impossible. More importantly, perhaps, Lorenz illustrates the rather generic notion that whilst initial uncertainty can sometimes be relatively unimportant, on other occasions it can rapidly destroy the accuracy of a prediction. This is where weather and climate change scientists may be in a position to pioneer techniques to predict and better communicate uncertainty in the evolution of complex nonlinear systems.

Uncertainty in the sense of inadequacy of our knowledge is present, for example, in current problems in cosmology, a unique science beset by special types of uncertainty. The emergence of a “dark universe”, so far unexplained, and the question of whether our universe is part of a much larger multiverse,  raise basic questions about uncertainties in our understanding of fundamental theories of physics.

Probabilistic approaches in policy making could be a more preferable way of sharing information compared to the traditional cautious focus on “worst case” scenarios.   For example, in recent government predictions of a flu pandemic, worst case scenarios were used for both policy making and public communication, leading to wide criticism of decisions made when the worst case did not materialise.

    Audio
    Mervyn King, Governor of the Bank of England, on uncertainty in macroeconomic policy making
    (3 mins).
  • Listen here.

While some may view probabilistic predictions as too complex for the public to understand, basic understanding may not be the key issue; the public understands that a horse  rated as 2-1 on is not a racing certainty to win, but can distinguish it from a 100-1 outsider.  If a more uniform approach to probabilistic prediction could be taken across a range of public-facing scientific disciplines, then acceptance of this approach by the public may be more widespread. For example, if the public were more exposed to weather prediction as a probabilistic forecast problem, this in turn might help dispel the false dichotomy of viewing the climate change problem either in terms of “belief” or “scepticism”.

Rupture caused by an earthquake in Mongolia in 1905.

Important political and business decisions are nevertheless made in the light of uncertain scientific input. Predictions which have properly quantified estimates of uncertainty make for better decision making than over-confident predictions with no estimate of uncertainty. But decisions can only be made if one can value the different probabilistic alternatives. In many business situations this may be a relatively simple economic matter. In other cases it is less straightforward. How do we value the sustainable existence of the Amazonian rainforest, or of the African Sahel? Estimating value, in this generalised sense, is clearly an extremely challenging issue for all of us. One of the consequences of the development of methodologies to estimate uncertainty is that it forces us to confront these difficult issues.

The problem of handling uncertainty is common across all disciplines, even if the language used varies markedly with discipline. Science may need to work more closely with the social science community on this.  Case studies where uncertainties have been assessed, both quantitatively and qualitatively, and yet decisions made in the face of these uncertainties, will help all scientists looking to estimate and communicate uncertainty.

Projection of trajectory of Lorenz system in phase space.

Discussion meeting

This article is based on the discussion meeting 'Handling uncertainty in science' which was held on 22-23 March 2010.

From the Royal Society's archives

The great 18th century mathematician and astronomer, Pierre-Simon Marquis de Laplace FRS, was a strong believer in an orderly and predictable universe. He imagined an intellect “which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed.”  Laplace then argued that his demon “would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.”

However, in the 19th century Scottish physicist James Clerk Maxwell FRS devised a very different kind of demon. He considered a hypothetical demon who could break the laws of thermodynamics. The upshot of Maxwell’s thought experiment was that what appeared to be a cast-iron law of nature only had a statistical certainty of being correct While this might sound fantastical (or at least statistically very unlikely), Maxwell’s thought experiment stimulated further scientific thought on statistical uncertainty.

Ultimately, the science of the 20th century destroyed belief in a predictable universe. The seminal work of Henri Poincaré (a President of the French Academy of Sciences), Ed Lorenz ForMemRS and Robert May FRS showed that even in classical mechanics the notion of a single Laplacian “formula” did not exist. Rather, the theory of Laplacian determinism became replaced by that of chaotic evolution, where the merest flap of a butterfly’s wings could make the difference between a storm hitting London or Paris, a month or so ahead.

In the 21st century we are still coming to terms with this radical paradigm shift. Although the Laplacian demon may be dead, useful predictions can still be made in our chaotic universe - as long as these predictions come with reliable estimates of uncertainty.