NCEP Global Ensemble: recent developments and plans Mozheng Wei*, Zoltan Toth, Dick Wobus*, Yuejian Zhu, Dingchen Hou* and Bo Cui* NOAA/NCEP/EMC, USA *SAIC at NOAA/NCEP/EMC 7 April 2005 2nd SRNWP Workshop on Short Range Ensemble Bologna,

1
NCEP Global Ensemble recentdevelopments and
plans Mozheng Wei, Zoltan Toth,Dick Wobus,
Yuejian Zhu, Dingchen Hou and Bo Cui
NOAA/NCEP/EMC, USA SAIC at NOAA/NCEP/EMC 7
April 2005 2nd SRNWP Workshop on Short Range
EnsembleBologna, Italy 7-8 April, 2005
2
OUTLINE

• NCEP GLOBAL ENSEMBLE SYSTEM
• A SUMMARY OF VARIOUS SCHEMES
• EXPERIMENTAL RESULTS
• DISCUSSION, NCEP 2005 PLAN AND OTHER RESEARCH
  ACTIVITIES

3
NCEP GLOBAL ENSEMBLE FORECAST SYSTEMTHE BREEDING
METHOD

• DATA ASSIM Growing errors due to cycling through
  NWP forecasts
• BREEDING - Simulate effect of obs by rescaling
  nonlinear perturbations (Toth and Kalnay
  1993,1997)
• (Zoltan Toth and Eugenia Kalnay started work
  in second half of 1991 Implemented in
  operational
• suite in December 1992 Upgraded system
  implemented in March 1994)
• Sample subspace of most rapidly growing analysis
  errors
• Extension of linear concept of Lyapunov Vectors
  into nonlinear environment
• Fastest growing nonlinear perturbations
• Not optimized for future growth

(Toth et. al 2004)
4
NCEP GLOBAL ENSEMBLE FORECAST SYSTEM (Toth et. al
2004)
RECENT UPGRADE (April 2003)
NEW CONFIGURATION MARCH 2004
10/50/60 reduction in initial perturbation size
over NH/TR/SH
FORMER SYSTEM
5
MOTIVATION FOR EXPERIMENTS

• EPS and DA systems must be consistent for best
  
• performance of both.
• DA provides best estimates of initial
  uncertainties, i.e. analysis
• error covariance, for EPS.
• EPS produces accurate flow dependent forecast
  (background)
• covariance for DA.

Best analysis error variances
EPS
DA
Accurate forecast error covariance
6
POSSIBLE SOLUTIONS

• CONSISTENCY can be achieved by
• (a) Development use of ensemble-based DA
  system
• Through THORPEX project, NCEP is collaborating
  with 4-5 groups on
• this. Istvan Szunyogh (Uni. of Maryland),
  Jeff Anderson (NCAR), Jeff
• Whitaker and Tom Hamill (NOAA/CDC) and Craig
  Bishop (NRL) and
• Milija Zupanski (Colorado State Uni.)
• (b) Coupling existing DA (3/4DVAR) with ensemble
  generation scheme
• Goal of present study
• As long as ensemble-based DA cannot outperform
  the 3/4DVAR,
• modify and couple existing DA and ensemble
  systems
• Simple initial perturbation scheme driven by
  analysis error variance
• from DA, 3/4DVAR driven by flow dependent
  forecast error covariance
• from ensemble

7
EXISTING/PROPOSED APPROACHES

• FIRST GENERATION INITIAL PERTURBATION GENERATION
  TECHNIQUES

PERTURBED OBSERVATIONS (MSC, Canada) BREEDING with Regional Rescaling (NCEP, USA) SINGULAR VECTORS with Total Energy (ECMWF)
ESTIMATION Realistic through sample, case dependent patterns amplitudes Fastest growing subspace, case dependent patterns No explicit estimate, not flow dependent
SAMPLING Random for all errors, incl. non-growing, potentially hurting short-range performance Nonlinear LVs, subspace of fastest growing errors Some dependence among perts. Directed, dynamically fastest growing in future, quite orthogonal.
CONSISTENCY BETWEEN ENS DA SYSTEMS Good quality of DA lagging behind 3DVAR? Not consistent, time-constant variance due to use of fixed mask Not consistent, potentially hurting short-range performance
8
EXISTING/PROPOSED APPROACHES - 2

• SECOND GENERATION INITIAL PERTURBATION GENERATOIN
  TECHNIQUES

ETKF, perts influenced by fcsts and observed data ET/BREEDING with Analysis Error Variance Estimate from DA Hessian Singular Vectors
ESTIMATION Fast growing subspace, case dependent patterns amplitudes Fastest growing subspace, case dependent patterns amplitudes Case dependent variance
SAMPLING Orthogonal in subspace of observations Orthogonal in analysis covariance norm Directed, dynamically fastest growing in future
CONSISTENCY BETWEEN ENS DA SYSTEMS Very good quality of DA lagging 4D-VAR. Good DAgtens EnsgtDA Climatologically consistent
9
COMPARISON OF DIFFERENT METHODS

• GRADUAL CONVERGENCE OF METHODS?
• Analysis error variance is commonly used in the
  2nd generation techniques.
• ETKF with no observation perturbation gt Breeding
  with orthogonalization and rescaling consistent
  with varying observational network
• COMMON CONCEPT
• Perturbations cycled dynamically through use of
  nonlinear integrations
• Bred Vectors (Toth Kalnay 1993) gt Nonlinear
  Lyapunov Vectors (Boffetta et. al 1998)
• Evolved SVs constrained by analysis error
  covariance (Hessian SVs) gt Finite-time Normal
  Mode (finite period) gt dominant Lyapunov vectors
  (longer time interval). (Wei Frederiksen 2004)
• COMMON CONCEPT With realistic initial
  constraint, evolved SV dynamics gt Lyapunov
  dynamics

10
DESCRIPTION OF 4 METHODS TESTED

• BREEDING with regional rescaling (Toth Kalnay,
  1993 1997)
• Simple scheme to dynamically recycle
  perturbations
• Variance constrained statistically by fixed
  analysis error estimate mask
• Limitations No orthogonalization fixed
  analysis variance estimate used.
• ETKF (Bishop et al. 2001 Wang Bishop 2003 Wei
  et. al 2004) used as perturbation generator
  (not DA)
• Dynamical recycling with orthogonalization in obs
  space
• Variance constrained by distribution error
  variance of observations
• Constraint does not work well with only 10
  ensemble members
• Issue of pert inflation is challenging for large
  variation of obs
• Computationally expensive
• Built on ETKF DA assumptions gt NOT consistent
  with 3/4DVAR
• Ensemble Transform (ET) (Bishop Toth 1999, Wei
  et. al 2005)
• Dynamical recycling with orthogonalization
  (inverse analysis error variance norm)
• Variance constrained statistically by fixed
  analysis error estimate mask
• Constraint does not work well with only 10
  ensemble members
• ET plus rescaling (Wei et al. 2005)

11
ET Formulation
12
EXPERIMENTS

• Time period
• Jan 15 Feb 15 2003
• Data Assimilation
• NCEP SSI (3D-VAR)
• Model
• NCEP GFS model, T126L28
• Ensemble
• 2x5 or 10 members, no model perturbations
• Evaluation
• 7 measures, need to add probabilistic forecast
  performance

13
Initial energy spread, Rescaling factor
distribution
?ET
?ETKF
?Breeding
?ETrescaling
14
Amp Factor?
Correlation
Effective Dim
15
Variance
PECA
16
?AC
?RMS error
17
SUMMARY OF RESULTS

• RMSE, PAC of ensemble mean forecast Most
  important
• ETRescaling and Breeding are best, ET worse,
  ETKF worst
• Perts and Fcst error correlation (PECA)
  Important for DA
• ETRescaling best, Breeding second
• Explained variance (scatterplots) Important for
  DA
• ET best
• Variance distribution (climatological,
  geographically)
• Breeding, ETRescaling reasonable
• Growth rate
• ETRescaling best? (not all runs had same initial
  variance)
• Effective degrees of freedom out of 5 members
• Minimal effect of orthogonalization
• Breeding (no orthogonalization) 4.6
• ET (built-in orthogonalization) 4.7
• Time consistency of perturbations (PAC between
  fcst vs. analysis perts)
• Important for hydrologic, ocean wave, etc
  ensemble forcing applications
• Excellent for all schemes, ET highest (0.999,
  breeding lowest, 0.988)
• New and very promising result for ET ETKF
• OVERALL hits out of 7

18
DISCUSSION

• All tests in context of 5-10 perturbations
• Testing with 80 members is under way
• Plan to experimentally exchange members
  with NRL
• (Will have total of 160 members)
• 4-Dim time-dependent estimate of analysis error
  variance
• Need to develop procedure to derive from
  SSI (GSI) 3DVAR
• ETRescaling looks promising
• Orthogonalization appears to help breeding
• Cheaper than ETKF, can also be used in targeting
• If ensemble-based DA can not beat 3/4DVAR
• Initial ens cloud need to be repositioned to
  center on 3/4DVAR analysis
• No need for sophisticated ens-based DA algorithm
  for generating initial
• perts?
• Good EPS
  Good DA

19
NCEP GLOBAL ENSEMBLE PLAN 2005
(Wei et. al 2005)
At every cycle, both ET and Simplex
Transformation (ST) are carried out for all 80
perts. Only 20 members are used for long fcsts.
ST is imposed on the 20 perts to ensure they
are centered around the analysis. 60 for short
6-hour fcsts.
41-60, ST 16-day fcsts
01-20, ST 16-day fcsts
21-40, ST 16-day fcsts
61-80, ST 16-day fcsts
time
00z
00z
06z
12z
18z
80-perts, ET,ST
80-perts, ET,ST
80-perts, ET,ST
80-perts,ET,ST
80-perts, ET,ST
20
OTHER RESEARCH ACTIVITIES AT NCEP

• REPRESENTING MODEL RELATED UNCERTAINTIES (D. Hou)
  
• (a) Experiments with multi-model version
  ensembles using different Cumulus
  Parameterization Schemes (CPS) ( accounts for
  little model uncertainty).
• (b) Experiments with varying the horizontal
  diffusion coefficient suggest that relatively
  strong diffusion in the current system hinders
  the increase in ensemble spread and leads to
  noticeable cold bias.
• (c) Stochastic physics schemes, using tendency
  difference between high/low resolution runs, or
  between two ensemble members to formulate the
  extra forcing term, resulted in systematic
  reduction in bias, sufficient spread, as well as
  moderate improvement in some performance scores.
• STATISTICAL POST-PROCESSING (reducing the
  biases, B. Cui)
• Adaptive, regime dependent Bias-Correction
  Algorithm (Kalman Filter type), applied to NCEP
  Operational Ensemble. It works well for first few
  days.
• (b) Climate mean bias correction (applied to
  CDC GFS Reforecast Data Set) can add value,
  especially for wk2 prob. fcsts.