Challenges in the Extraction of Decision Relevant Information from Multi-Decadal Ensembles of Global Circulation Models

1
Challenges in the Extraction of Decision Relevant
Information from Multi-Decadal Ensembles of
Global Circulation Models
Dave Stainforth Acknowledgements A. Lopez. F.
Niehoerster, E. Tredger, N. Ranger, L. A. Smith
Grantham Research Institute Centre for the
Analysis of Timeseries, London School of
Economics and Political Science
Climate Change Workshop Statistical and Applied
Mathematical Science Institute 18th February 2010

1. Introduction and context.
2. The difficulties in predicting climate.
3. Domains of possibility.
4. Metrics.
5. Implications for future experiments.
6. Transfer functions

2
Introduction

• Climate models can help us
• understand the physical system.
• generate plausible storylines for the future.
• build better models.
• Context
• responding to societal desire for predictions of
  the impacts of climate change
• providing information to guide climate change
  adaptation strategies.
• minimise vulnerability/maximise resilience
  .vs. predict and optimise
• International adaptation when is adaptation
  adaptation and when is it development?
• More uncertainty, please.

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
Climate Prediction A Difficult Problem

• A problem of extrapolation
• Verification / confirmation is not possible.
• Model deficiencies
• Model inadequacy they dont contain some
  processes which could have global impact.
  (methane clathrates, ice sheet dynamics, a
  stratosphere, etc.)
• Model uncertainty Some processes which are
  included are poorly represented e.g. ENSO,
  diurnal cycle of tropical precipitation.
• Model interpretation
• Lack of model independence.
• Metrics of model quality
• Observations are in-sample.
• Ensembles are analysed in-sample.
• Models which are bad in some respects may contain
  critical feedbacks in others.
• Non-linear interactions selecting on a subset of
  variables denies the highly non-linear nature of
  climatic interactions.

7
Types of Climate Uncertainty

• External Influence (Forcing) UncertaintyWhat
  will future greenhouse gas emissions be?
• Initial Condition Uncertainty(partly aleatory
  uncertainty)The impact of chaotic behaviour.
• Model Imperfections(epistemic uncertainty)Differ
  ent models give very different future projections.

Figure IPCC AR4
8
Uncertainty Exploration
Type of Uncertainty Response
Forcing Uncertainty Ensembles of Emission scenarios
Initial Condition Uncertainty Initial Condition Ensembles (ICEs). (V. small. Typically max of 4 sometimes 9)
Model Deficiencies. Multi-model ensembles e.g. CMIP III O(10) Perturbed-parameter ensembles - O(10000-100000) climateprediction.net - O(100) in-house teams e.g. MOHC
9
Climate Prediction A Difficult Problem

• A problem of extrapolation
• Verification / confirmation is not possible.
• Model deficiencies
• Model inadequacy they dont contain some
  processes which could have global impact.
  (methane clathrates, ice sheet dynamics, a
  stratosphere, etc.)
• Model uncertainty Some processes which are
  included are poorly represented e.g. ENSO,
  diurnal cycle of tropical precipitation.
• Model interpretation
• Lack of model independence.
• Metrics of model quality
• Observations are in-sample.
• Ensembles are analysed in-sample.
• Models which are bad in some respects may contain
  critical feedbacks in others.
• Non-linear interactions selecting on a subset of
  variables denies the highly non-linear nature of
  climatic interactions.

10
Consequences of Lack of Independence 1
See Stainforth et al. 2007, Phil Trans R.Soc A
Climateprediction.net data
11
Consequences of Lack of Independence 2
From Stainforth et al. 2005
12
Can Emulators Help Out Here? No

• Even the shape of model parameter space is
  arbitrary so filling it in does not help in
  producing probabilities of real world behaviour.

13
An Aside UK Climate Projections 2009 (UKCP09) - 1
Change in mean summer precip 10 90
Murphy et al, 2004
UKCIP, 2009
14
An Aside UK Climate Projections 2009Change in
Wettest Day in Summer Medium (A1B) scenario
2080s 90 probability levelvery unlikely to be
greater than
2080s 67 probability levelunlikely to be
greater than
15
An Aside A (Very) Basic Summary of My
Understanding of the Process

• sample parameters,
• run ensemble,
• emulate to fill in parameter space,
• weight by fit to observations

Emulate
16
An Aside Issues

• Size of ensemble given size of parameter space.
• The ability of the emulator to capture non-linear
  effects.
• The choice of prior i.e. how to sample parameter
  space.
• The justification for weighting models.
• On what scales do we believe the models have
  information?

17
Choices of Model Parameters

• Most model parameters are not directly
  representative of real world variables. e.g. the
  ice fall rate in clouds, the entrainment
  coefficient in convection schemes.
• Their definition is usually an ad hoc choice of
  some programmer. (Possibly a long time ago, in a
  modelling centre far away.)
• Thus a uniform prior in parameter space has no
  foundation and
• testing the importance of such a prior is not a
  matter of tweaks around the edges (adding 15 to
  the limits, or exploring a triangular prior
  around central values)
• rather it is a matter of sensitivity to putting
  the majority of the prior points in one region

18
An Aside Issues

• Size of ensemble given size of parameter space.
• The ability of the emulator to capture non-linear
  effects.
• The choice of prior i.e. how to sample parameter
  space.
• The justification for weighting models.
• On what scales do we believe the models have
  information?

Choice of parameter definition
19
Estimated distributions for climate sensitivity
upper bounds depend on prior distribution
Uniform prior in sensitivity
Uniform prior in feedbacks
Frame et al, 2005
20
Climate Prediction A Difficult Problem

• A problem of extrapolation
• Verification / confirmation is not possible.
• Model deficiencies
• Model inadequacy they dont contain some
  processes which could have global impact.
  (methane clathrates, ice sheet dynamics, a
  stratosphere, etc.)
• Model uncertainty Some processes which are
  included are poorly represented e.g. ENSO,
  diurnal cycle of tropical precipitation.
• Model interpretation
• Lack of model independence.
• Metrics of model quality
• Observations are in-sample.
• Ensembles are analysed in-sample.
• Models which are bad in some respects may contain
  critical feedbacks in others.
• Non-linear interactions selecting on a subset of
  variables denies the highly non-linear nature of
  climatic interactions.

21
Domains of Possibility 1
From Stainforth et al. 2005
22
Domains of Possibility 2
See Stainforth et al. 2007, Phil Trans R.Soc A
Climateprediction.net data
23
Climate Prediction A Difficult Problem

• A problem of extrapolation
• Verification / confirmation is not possible.
• Model deficiencies
• Model inadequacy they dont contain some
  processes which could have global impact.
  (methane clathrates, ice sheet dynamics, a
  stratosphere, etc.)
• Model uncertainty Some processes which are
  included are poorly represented e.g. ENSO,
  diurnal cycle of tropical precipitation.
• Model interpretation
• Lack of model independence.
• Metrics of model quality
• Observations are in-sample.
• Ensembles are analysed in-sample.
• Models which are bad in some respects may contain
  critical feedbacks in others.
• Non-linear interactions selecting on a subset of
  variables denies the highly non-linear nature of
  climatic interactions.

24
Best Information Today / Best Ensemble Design For
Tomorrow

• For tomorrow Design ensembles to push out the
  bounds of possibility.
• For today Use the best exploration of model
  uncertainty combined with the best global
  constraints.

25
Issues/Questions in Ensemble Design to Explore
Uncertainty

• Emulators to guide where to focus parameter space
  exploration.(Potentially very powerful in
  distributed computing experiments.)How?
• Simulation management to minimise the consequence
  of in-sample analysis.How?
• Questions of how we describe model space to
  enable its exploration.
• How do we evaluate the spatial and temporal
  scales on which a model is informative?
• How do we integrate process understanding with
  model output in such a multi-disciplinary field.
• How do we integrate scientific information with
  other decision drivers.
• Better understanding and description of the
  behaviour non-linear systems with time dependent
  parameters.
• How do we evaluate information content?

26
Resolution .vs. complexity .vs. uncertainty
exploration

• What processes do we need to include in our
  models?
• What do we need our models to do to answer
  adaptation questions?
• What would be the perfect ensemble?
• What should be the next generation ensemble?

27
Lets Be Careful Out There