Predictability, Ensemble Forecasts, and the use of Statistical Guidance in the Forecast Process

1
Predictability, Ensemble Forecasts, and the use
of Statistical Guidance in the Forecast Process

• Steve Keighton
• National Weather Service
• Blacksburg, VA

2
Outline

• Chaos theory and predictability in the atmosphere
• Numerical Weather Prediction (NWP) and use of
  ensemble forecast methods
• Use of statistical guidance in the forecast
  process (Model Output Statistics) if time

Acknowledgments

• Josh Korotky NWS Pittsburgh
• Mark Antolik NWS Meteorological Development Lab

3
Prediction is very difficult, especially about
the future - Niels Bohr
4
Prelude What is Chaos and why is it important?

• Chaos leads us from the laws of nature to their
  consequences
• shows us that simple systems can exhibit complex
  behaviorand vice versa
• demonstrates that unpredictable behavior can
  develop in a system governed by deterministic
  laws
• As forecasterschaos shows us the limits of
  predictability
• highlights the importance of probabilistic
  thinking
• shows us the value of expressing uncertainty in
  forecasts
• helps us understand why the future of
  forecasting will lean heavily on ensemble rather
  than deterministic approaches

5
Edward Lorenz (1917 2008)

• Small errors in the initial state estimate of a
  nonlinear system can limit the prediction of
  later states of the system
• Chaos occurs when error propagation, seen as a
  signal in time, grows to the same size or scale
  as the original signal...

one flap of a sea-gulls wing may forever
change the future course of the weather (Lorenz,
1963)
6
Elements of Chaos

• Dynamical system future states caused by past
  states (determinism)
• Nonlinearity system output (response) isn't
  proportional to input (forcing)
• a small forcing can lead to a disproportionately
  large response and vise versa
• a system's values at one time arent proportional
  to the values at an earlier time
• Non-periodic behavior future states never
  repeat past states
• Extreme sensitivity to initial conditions small
  initial state uncertainties amplify
• a "prediction horizon is inevitable
• Even though the governing laws of a system are
  known, long-term predictions can be meaningless
• Chaos occurs only in deterministic, nonlinear,
  dynamical systems

7
Attractors General Statements

• An attractor is a dynamical system's set of
  conditions
• In a phase space diagram, an attractor shows a
  system's long-term behavior. It's a compact,
  global picture of all of a system's possible
  steady states.
• All attractors are either nonchaotic or chaotic
• Nonchaotic attractors generally are points,
  cycles, or smooth surfaces (tori), and have
  regular, predictable trajectoriessmall initial
  errors or minor perturbations generally don't
  have significant long-term effects
• Chaotic or strange attractors occur only after
  the onset of chaos. Long term prediction on a
  chaotic attractor is limitedsmall initial errors
  or minor perturbations can have profound
  long-term effects

8
Strange Attractors

• A Strange Attractor is dynamically unstable and
  non periodic
• - A chaotic system is unstableits behavior
  changes with time rather than settling to a fixed
  point
• - Chaotic systems are non periodictrajectories
  do not settle into repeatable patterns and never
  cross
• - A chaotic attractor shows extreme sensitivity
  to initial conditions trajectories initially
  close, diverge, and eventually follow very
  different paths

9
Difference between linear and non-linear systems
Behavior over time. Linear processes are smooth
and regular, whereas nonlinear ones may be
regular at first but often change to
erratic-looking. Response to small changes in
the environment or to stimuli. A linear process
changes smoothly the response of a nonlinear
system is often much greater than the stimulus.
Persistence of local pulses. Pulses in linear
systems decay over time. In nonlinear systems,
pulses can persist for long times, perhaps
forever.
10
Non-periodic Dynamical System

• A dynamical system that never settles into a
  steady state attractor
• Non periodic systems never settle into a
  repeatable (predictable) sequence of behavior.
• Prediction of a future state of a non periodic
  system is eventually impossible, due to nonlinear
  dynamics (feedback)
• The atmosphere illustrates non periodic behavior
• Broad patterns in the development, evolution, and
  movement of weather systems may be noticeable,
  but no patterns ever repeat in an exact and
  predictable sequence
• The atmosphere is
• damped by friction of moving air and water
• driven by the Suns energy
• the ultimate feedback system
• Weather patterns never settle into a steady state
  attractor

11
Sensitivity to Initial Conditions

• Small uncertainties (minute errors of measurement
  which enter into calculations) are amplified
• Result system behavior is predictable in the
  short termunpredictable in the long term

12
The Lorenz Discovery

• From nearly the same starting point (tiny
  rounding error), the new forecast diverged from
  the original forecasteventually reaching a
  completely different solution!
• Why? Slight differences in the initial
  conditions had profound effects on the outcome of
  the whole system
• Lorenz found the mechanism of deterministic
  chaos simply-formulated systems with only a few
  variables can display highly complex and
  unpredictable behavior

(.506) vs. (.506127) Initial condition
13
Chaos and Numerical Weather Prediction (NWP)
If a process is chaotic knowing when reliable
predictability dies out is useful, because
predictions for all later times are useless.

• Weather forecasts lose skill because of
• Chaos small errors in the initial state of a
  forecast grow exponentially
• Model uncertainty
• Numerical models only approximate the laws of
  physics (important small scale processes are
  parameterized)
• Very small errors in the initial state of a
  forecast model grow rapidly at small scales, then
  spread upscale
• Forecast skill varies both spatially and
  temporally as a result of both initial state and
  model errors, which change as the atmospheric
  flow evolves

14
Models must simulate numerous irresolvable
processes
15
NWP Skill as a Function of Scale and Time
Days 6-7
Days 3-5
Days 1-2
lt Day1
Feature/Variable
Days 6-7
Days 3-5
Days 1-2
lt Day1
Feature/Variable
Good
Very Good
Excellent
Excellent
Hemispheric flow transitions
Good
Very Good
Excellent
Excellent
Hemispheric flow transitions
Low skill-Fair
Fair-Good
Very Good
Excellent
Cyclone life cycle
Low skill-Fair
Fair-Good
Very Good
Excellent
Cyclone life cycle
----
Fair
Good
Excellent
Fronts
----
Fair
Good
Excellent
Fronts
----
----
Fair
Good
Mesoscale banded structures Convective clusters
----
----
Fair
Good
Mesoscale banded structures Convective clusters
Skill with max/min
Very Good
Excellent
Temp / wind
Skill with max/min Temp
Very Good
Excellent
Temp / wind
Some skill in 5-10 day QPF
Good
Very Good
QPF/ mean clouds
Fair
Good
Very Good
Precip/ mean clouds

• Predictability falls off as a function of scale
• Large scale features (planetary waves) may be
  predictable up to a week in advance
• Small systems (fronts) are well forecasted to day
  2.. cyclonic systems to day 4

16
Coping with NWP Predictability

• The largest obstacles to realizing the potential
  predictability of weather and climate are
  inaccurate models and insufficient
  observationsan intrinsic limit of
  predictability will always exist however
• In the last 30 years, most improvements in
  weather forecast skill have come from
  improvements in models and data assimilation
  techniques
• Even though increased resolution increases error
  growth rates, increased resolution more
  accurately simulates physical processes, allowing
  more accurate scale interactions and forecast
  evolution
• High resolution ensembles represent the best of
  both worlds
• Added realism of high-resolution
• An attempt to account for the inherent
  uncertainties

17
How do Ensembles help us cope with Chaos?
18
Why cant we count exclusively on single model
NWP?

• Overlooks forecast uncertainty
• Initial condition and model uncertainty
• Chaotic flows vs. stable flow regimes
• Potentially misleading
• Oversells forecast capability

19
GFS 84 hr forecastValid 00Z 22 Nov
NAM 84 hr forecastValid 00Z 22Nov
Single Model NWP
Which model do you believe?
20
Ensembles and PDF

• Recognizing the eventuality of chaosweather
  forecasts can provide more useful information by
  describing the time evolution of an ensemble
  probability density function (PDF)
• Initial PDF represents initial uncertainty
• Single forecast doesnt account for initial and
  model erroroften fails to predict the real
  future state past a certain point
• Ensemble of perturbed forecasts accounts for
  initial and model error PDF of solutions more
  likely to contain real future state
• Ensemble PDF contains additional information,
  including forecast uncertainties

21

• Produce a series of forecasts, each starting from
  slightly different initial conditions and/or
  model formulations
• Properties of the forecast PDF offer important
  probabilistic and statistical informationthe
  spread of forecast trajectories quantifies
  forecast uncertainty
• if the initial perturbations sampled the errors
  of the day
• and the ensuing ensemble spread captures the
  forecast errors

22
Ensemble Prediction System (EPS) Goals

• Represent initial condition and/or model
  uncertainty
• Determine a range of possible forecast outcomes
• Estimate the probability for any individual
  forecast outcome
• General provide a framework for decision
  assistance

23
General EPS forecasting tools

• Spaghetti Plots (shows all solutions)
• Mean/Spread (middleness and variability)
• Probabilities
• Most Likely Event

24
Spaghetti Plots
25
Mean and Spread

• Characteristics of mean
• The ensemble mean performs better on average than
  operational model on which it is based. Why?
• Because predictable features remain intact, less
  predictable features are smoothed out
• Characteristics of spread
• Allows assessment of uncertainty, since more
  spread means more uncertainty

4
1
3
2
26
Probability of Exceedance

• Helps determine the probability of a specified
  event.
• Gives probability of exceeding meaningful
  threshold
• Calculation represents count of what of
  ensemble members exceed the threshold of interest
• Example here is for 12-hour precipitation
  exceeding 0.25 inches.

27
Most Likely or Dominant Event Diagram

• Used to show what is most often predicted by the
  ensemble forecast
• A common example
• Precipitation type (snow, sleet, freezing rain,
  rain)

28
Summary

• Chaos and model uncertainties impose a very real
  physical limit on predictability
• Predictability falls off (sometimes rapidly) as a
  function of scale and time
• Forecast accuracy varies both spatially and
  temporally as a result of initial state and model
  errors, which change as the atmospheric flow
  evolves
• Ensemble NWP optimizes predictability for all
  scales, and extends the utility of
  forecastsespecially at extended ranges (days
  4-7)
• Allows for quantification of uncertainty, and
  foundation for decision assistance

29
Statistical Guidance in the Forecast Process
30
WHY STATISTICAL GUIDANCE?

• Add value to direct NWP model output
• Objectively interpret model
• - remove systematic biases
• - quantify uncertainty
• Predict what the model does not
• Produce site-specific forecasts
• (i.e. a downscaling technique)
• Assist forecasters
• First Guess for expected local conditions
• Built-in model/climatology

31
MODEL OUTPUT STATISTICS (MOS)
Relates observed weather elements (PREDICTANDS)
to appropriate variables (PREDICTORS) via
a statistical approach. Predictors are obtained
from

• Numerical Weather Prediction (NWP) Model
• Forecasts
• 2. Prior Surface Weather Observations
• 3. Geoclimatic Information
• Current Statistical Method
• MULTIPLE LINEAR REGRESSION
• (Forward Selection)

32
MODEL OUTPUT STATISTICS (MOS)
Properties

• Mathematically simple, yet powerful
• Need historical record of observations
• at forecast points
• (Hopefully a long, stable one!)
• Equations are applied to future run of
• similar forecast model
• Probability forecasts possible from a
• single run of NWP model
• Other statistical methods can be used
• e.g. Polynomial or logistic regression
• Neural networks

33
MODEL OUTPUT STATISTICS (MOS)

• ADVANTAGES
• - Recognition of model predictability
• - Removal of some systematic model bias
• - Optimal predictor selection
• - Reliable probabilities
• - Specific element and site forecasts
• DISADVANTAGES
• - Short samples
• - Changing NWP models
• - Availability quality of observations

34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
Gridded MOS

• MOS at any point (GMOS)
• - Support NWS digital forecast database
• 2.5 km - 5 km resolution
• - Equations valid away from observing sites
• - Emphasis on high-density surface networks
• - Use high-resolution geophysical data
• - Some problems over steep terrain or
  data-sparse
• regions

42
Gridded MOS
43
Use of MOS at a Forecast Office

• Can ingest GMOS directly into local digital
  forecast database
• Can apply bias correction (based on performed
  in past 30 days)
• Can ingest point-based MOS and spread it to
  entire grid
• MOS from single models or from ensemble
  mean/max/min
• We verify our forecast against MOS, so we may
  use as a starting point but we try to improve on
  it based on local experience or recent trends

44
Questions?