A Stochastic Perturbation Scheme For Representing Model Related Uncertainty Dingchen Hou, Zoltan Tot

1
A Stochastic Perturbation Scheme For
Representing Model Related UncertaintyDingchen
Hou, Zoltan Toth and Yuejian Zhu
Acknowledgements Mark Iredell, Henry Juang,
Stephane Vannitsem,Richard Wobus, Bo Cui, Cecile
Penland, Prashant Sardeshmukh,Weiyu Yang, James
Purser and Mozheng WeiNOAA THORPEX PI
WORKSHOPJanuary 17-19, 2006 at NCEP, Camp
Springs, MD
2
OUTLINE

• MOTIVATION
• STOCHASTIC PARAMETERIZATION SCHEMES
• Existing Schemes
• The Proposed Scheme
• A Simplified Version
• RESULTS OF EXPERIMENTS
• Outliers
• Spread and Ensemble Mean Forecast
• Probabilistic Forecasts
• Comparison with a Bias-correction
  Procedure
• Combination with a Bias-correction
  Procedure
• SUMMARY AND ONGOING RESEARCH

3
Motivation

• Why do we need ensemble forecasting
• To represent forecast uncertainty.
• Sources of Forecast Uncertainty
• INITIAL CONDITIONS
• MODEL
• Approaches to Representing Model Related
  Uncertainties
• a) Multiple Model
• Multi-version of a single model (e.g.
  Houtekamer et al. 1996 Stensrud et al. 2000 Hou
• et al. 2004 Du, 2004)
• Multi-model, Multi-version (e.g. MSC
  global ensemble Du and Tracton 2001)
• Multi-model, Multi-center (e.g.
  Richardson, 2001 THORPEX NAEFS)
• b) Stochastic Parameterization
• Perturbation Rescaling (Toth and Kalnay
  1995)
• -------- Multiplicative
• Stochastic Physics of ECMWF (Buizza et
  al. 1999)
• -------- Additive, perturbing
  the parameterized tendency
• Spatiotemporal noise (Perez-Munuzuri et
  al. 2003)

4
Existing Stochastic Perturbation Schemes
General Framework X is a model variable T is
the conventional tendency S is the stochastic
tendency term.
Model equation, conventional form
Model equation, stochastic parameterization
For Ensemble Forecast
i1,2,,N for N members 0 for control

• How to Formulate the Stochastic Forcing S?
• ECMWF (Buizza et al. 1999) S is related to
  parameterized component of T, multiplied by a
  random number. Spatially/temporally discontinous.
• Shutts, 2004 Stochastic Kinetic Energy
  Backscatter (SKEB). S is related to KE
  dissipation rate. A cellular automation is used
  to generate evolving patterns. Only applied to
  the stream function field. Not balanced.
• Mylne et al, 2005 Stochastic Convective
  Vorticity (SCV) and Random Parameters (RP). S is
  implicitly related to the CAPE or physics
  parameterization parameters.
• Toth and Kalnay, 1995 and Perez-Munuzuri et al
  (2003) S State perturbations
• Perez-Munuzuri et al (2003) S is not related to
  physical processes/variables.

5
Formulation ofthe Proposed Stochastic
Perturbation Scheme
Ensemble Forecast
i1,2,,N for N members 0 for control
Required properties of the Stochastic Forcing
Terms S 1. Forcing applied to all
variables 2. Approximately balanced
3. Smooth variation in space and time
4. Flow dependent 5.
Quasi-orthogonal
Assumption The the ensemble perturbations of the
conventional tendencies ( ), denoted as
, provide a sample of
realizations of the stochastic forcing S.
P vectors are quasi orthogonal and vary
smoothly in space and time.
Strategy Generate the S terms from (random)
linear combinations of the conventional
perturbation tendencies, i.e.
Generalization of the Toth and Kalnay (1995)
method Similar to ET but applied to
ensemble perturbation tendencies successively

• Matrix Notation S (t)
  P(t) W(t)
• MxN MxN NxN
• For an ensemble of N forecasts at M grid points.
• As P is quasi orthogonal, an orthonormal matrix
  W ensures orthogonal S.
• M is also required to have smooth temporal
  variation.

6
Formulation ofthe Proposed Stochastic
Parameterization SchemeDetermination of
Combination coefficients

• Initialize W W(t0)
• An orthonormal matrix W (an
  orthonormal basis in N-Dimensional space) of
  dimension NxN, can be generated by independently
  sampling random numbers (from a gauss
  distribution) and then applying the Gramm-Schmidt
  procedure.
• 2) Rotate W W(t) W(t-1) R(t)
• Random linear transformation is
  used to make the matrix W a function of time.
  R(t) is an orthonormal matrix with its diagonal
  elements close to 1 and others small, generated
  by applying the Gram-Schmidt procedure to a
  random anti-symmetric matrix. W(t) remains
  orthonormal. (suggested by J. Purser)

Plotted are time series of the combination
coefficients w(i,j), j1,2,,10, for i1. .
7
Formulation ofA simplified version
1. Use a single perturbation tendency instead of
a combination
Random match between i and j
2. Use finite difference form for the stochastic
term
For tk6hr, where k1,2,3, and
otherwise

• as represent coefficients used to
  re-scale the perturbations to a representative
  size in NH, SH and TR, using 500hPa kinetic
  energy as the norm, and

8
Vorticity Increment
Example of the Stochastic Forcing 500hPa
vorticity increment and corresponding total
kinetic energy increment at 18h of forecast,
initialized 00Z, Sep. 25, 2004.
500 hPa height, 00Z, 09/26/2004
Kinetic Energy Increment
9
Outliers H500, day 6 forecast, 20041002
Without SP large number of outliers with
negative and positive forecast bias
With SP the number of outliers is significantly
reduced
10
Statistics Percentage Excessive Outliers,
reduced by SP
Forecast consistency improved by SP
11
Statistics Ensemble Spread and Error of Ensemble
MeanIncreased Spread, Reduced Mean Error
(ME)Reduced Mean Absolute Systematic Error
(MASE)----- Without SP ------- With SP
Solid, rmse Dash spread
Solid rmse Dash spread
MASE
Mean Error
Mean Error
12
Comparison with Post-Processing (Cui,Toth and
Zhu,2005)Stochastic Parameterization (SP)
Bias-Correction (PP) In MASE
reduction, SP is effective in week 2
---- Without PP ---- With 1PP
rms
---- Without SP ---- With SP
rms
Spread
Spread
MASE
MASE
ME
ME
13
Comparison with Post-Processing (PP) RPSS
Improved in both cases (SP and PP)SP is more
effective in week 2 forecast
Stochastic Parameterization (SP)
Bias-correction (1PP)
---- Without SP ---- With SP ---- Without SP
but optimal pp (upper limit)
---- Operation ---- Operation 1PP ----
Operation optimal pp (upper limit)
14
Comparison with Post-Processing (PP) BSSSP
increases BSS by reducing its reliability
componentStochastic Parameterization (SP)
Bias-Correction (PP)
BSS
BSS
---- Without SP ---- With SP ---- Without SP
but optimal pp
---- Operation ---- Operation 1PP ----
Operation optimal pp
Reliability
Reliability
Resolution
Resolution
15
Combination of SP and PPInsert the 2004 October
experiments forecast into operational data
setsto apply the adaptive algorithm of Bias
Correction (PP)For RPSS Score, the positive
impact of SP and PP adds up, leading to
improvement for all lead times and at greater
extend ! Performance of SPPP is highest
although the procedure may underestimate its
score due to over-correction
rms
---OPT ---SP ---OPTPP ---SPPP
--- Without SP --- With SP --- Without SP
after PP --- With SP after PP
Spread
RPSS
ME
16
Understanding the Impact of the SP scheme on the
System

• Benefits of Stochastic parameterization Expected
  (Palmer, 2003) and
• Realized with the current scheme
• More Complete representation of model
  uncertainty
• Increase spread
• Reduction in model systematic error
  (noise-induced drift)
• Significant reduction in ME and MASE
• Improvement in Probabilistic
  forecast scores
• More Accurate estimate of internal climate
  variability
• More improvement in week 2 than in
  week 1?

Palmer, 2003
Mean state with noise
Mean state without noise
17
SUMMARY

• A stochastic parameterization scheme is
  proposed and tested for NCEP Global Ensemble
  Forecast System with a simplified version. The
  structured stochastic forcing terms are based on
  the differences in tendencies between ensemble
  members and the control run.
• The stochastic forcing fields added are
  balanced, flow-dependent and have random noise
  structure and geographic patterns, changing
  smoothly in time and space.
• By including this stochastic
  parameterization scheme, the ensemble spread is
  significantly increased, with fewer outliers. The
  Mean Error and Mean Absolute Systematic Error
  (MASE) of the ensemble mean forecast is also
  significantly reduced. For probabilistic
  forecasts, the scheme can significantly improve
  performance scores. The scheme is more effective
  in week 2 forecast compared with week 1 forecast.
• Combining the stochastic
  parameterization scheme with a post processing
  procedure (adaptive Bias-Correction) will further
  improve the forecast, leading to significant
  improvement at all lead times. This suggests
  that the stochastic parameterization scheme
  really make difference!

18
ONGOING RESEARCH

• Modify scripts and code to test the
  scheme without simplifications.
• Conduct experiments to test the scheme
  for longer period and various seasons.
• Verify the forecast output for more
  variables and verification scores.
• Develop software capacity to synchronize
  the integration of the ensemble members and
  control run (ESMF project, Weiyu Yang) so that
  the scheme can be implemented for operation.