Update to COPC:

1

Update to COPC Global Model Performance
Dropouts
Dr. Jordan Alpert NOAA Environmental Modeling
Center contributions from Dr. Brad Ballish, Dr.
Da Na Carlis, Dr. Rolf Langland and CDR Mark Moran
2
Outline

• Tools to compare ECMWF and NCEP Dropouts
• Model Runs for Dropout Experiments
• Results from SH Dropout Experiments
• Model Runs for Dropout Experiments
• Comparison of How Well the GSI and ECM Fit Obs
• Wind Speed Biases
• How GSI and ECM Fit Observations

3
Poor Forecasts or Skill Score Dropouts Lower
GFS Performance.
Detailed on next slide
4
Skill Score Dropouts Lower Overall GFS
Performance.
5
Analysis Team for Dropout Issue

• Jordan Alpert - EMC
• DaNa Carlis - EMC
• Brad Ballish - NCO
• Krishna Kumar - NCO
• Rolf Langland - NRL

6
Tools to compare ECMWF and NCEP Dropouts

• Use ECMWF analysis to generate Pseudo Obs for
  input to the Gridded Statistical Interpolation
  (GSI) and GFS forecasts.
• Generate a Climatology of NH and SH dropouts
  what are the systematic differences
• Interpolate ECM and GSI analyses to observations
  to determine comparative strengths and
  weaknesses.
• Statistically analyze observation type fits
• stratify by pressure, type, and difference
  magnitude

Goal Diagnose Quality Control problems to
implement real-time QC detection/correction and
improvements to analysis system algorithms
7
Model Runs for Dropout Experiments

• GFS Operational (Production) GFS
• ECMWF Operational
• ECM Pseudo Obs from ECMWF analysis
  using the GSI as a Grand Interpolator
• ecmanlGES GSI run using ECM as background guess
  plus GDAS observations
• CNTRL Updated GSI system with GDAS obs
• InterpGES Updated GSI with previous 3, 6,
  9-hour
• ECM forecasts as
  background guess plus
• GDAS observations

8
Results from SH Dropout Experiments

• Using ECM (Pseudo Obs) yielded a 90 success rate
  for alleviating SH dropouts.
• Adding GDAS Obs (ecmanlGES) lowers the success
  rate to 75.
• Combining updated GSI system / GDAS obs (CNTRL)
  with all obs improves GSI but not enough to
  alleviate dropouts
• Combining background guess from ECM plus all obs
  (InterpGES ) also improves GSI but not enough to
  alleviate dropouts
• Composites show low-level temperature difference
  near 850mb in SH.
• Adding the SH observations and the 3, 6, 9-h
  background guess degrades the ECM forecast

9
Rolf Langland (NRL Monterey) shows systemic
height differences between all models and ECMWF
(shown is ECMWF-NCEP). Cause may be the
difference in satellite window coverage (under
study) ECMWF (12-h) vs. others (6-h).
Plots at left show height difference plots of
time (October to December 2007) vs. longitude,
averaged over 35-65N latitudes. The range of
the bias is 12 m
10
Comparison of How Well the GSI and ECM Fit
Observations

• Statistics are made on how well the GSI and ECM
  analyses fit the observations for different
  observation types, as function of pressure, for
  different regions
• This study shows that each analysis fits
  certain observation types with differing amounts
  of success, but does not conclude which performs
  better.

11
Comments on Wind Speed Biases

• The ECM analysis winds at satellite wind
  locations are stronger than those of the GSI
• The ECMWF data quality control is more aggressive
  at deleting satellite winds with speeds slower
  than the background
• ECM wind speeds are stronger as weaker satellite
  winds are deleted or given less weight in the
  ECMWF analyses
• For ACARS and sondes, the biases are similar

Work continues to analyze satellite wind speed
biases to determine an implementable algorithm
for QC, bias correction, and analysis weights
12
How GSI and ECM Fit Observations

• The GSI is found to draw more closely for most
  winds, especially satellite winds and when there
  are moderate differences in the analyses
  (outliers)
• The ECM analyses draw more closely for radiosonde
  temperatures, especially away from the middle
  atmosphere and for moderate differences in the
  analyses
• Fits for surface pressure and moisture are
  similar not shown here
• ECM winds are stronger at satellite wind
  locations based on speed bias stats but with
  similar fits for radiosonde and ACARS speeds
• Changes in the QC and Ob errors could retune
  these data fits

13
Summary and Future Work

• ECM analysis show dropouts can be alleviated in
  GFS forecasts
• Running the operational GSI with an ECM derived
  background guess results in better forecast skill
  than the operational GFS but not as good as ECM
  runs
• Running the operational GSI after removing select
  observation types offers a systematic approach
  for assessing the impact of different observation
  types
• There are systematic height differences between
  all models and ECMWF perhaps from the larger
  satellite window coverage of ECMWF (12-h) vs
  others (6-h)
• Work continues to analyze what is the optimal fit
  of the analysis to observation types and to
  determine an implementable algorithm for improved
  QC, bias correction, and analysis weights
• Work has started to use baroclinic instability
  rates to help analyze dropout cases and analysis
  differences

14
Background Slides
15
ECMWF INITIAL CONDITIONS FOR GFS FORECASTS ECM
Runs
ECM(WF) Analysis 1x1 deg, 15 levels --------------
----------- PSEUDO ECM OBS
ECM ANL ------------ GFS FORECAST
PSEUDO ECM OBS GFS GUESS -------------- ECM
ANLYSIS PRE-COND GUESS
Run GSI
PRE-COND ECM GUESS PSEUDO ECM OBS ------------
--- ECM ANL
(
)
INPUT ----------- OUTPUT
Run GSI again
16
InterpGES Analysis ------------ GFS
FORECAST 5-days
InterpGES Runs
ECM(WF) Analysis -------------------------
PSEUDO ECM OBS
GFS FORECAST BKGND GES -3, 0, 3 GDAS Obs
------------ InterpGES Analysis
ECM ANL ------------ GFS FORECAST 3, 6, 9-hour
PSEUDO ECM OBS GFS GUESS -------------- ECM
ANLYSIS PRE-COND GUESS
Run GSI
Run GSI again
PRE-COND ECM GUESS PSEUDO ECM OBS ------------
--- ECM ANL
(
)
INPUT ----------- OUTPUT
17
Excerpt of SH Dropout Climatology List (AC skill
scores)
NB 90 ECM Success in alleviating SH dropout.
Adding GDAS Obs (ecmanlGES) lowers the success
rate to 75. GSI upgrades (CNTRL) improves GSI
but not enough to alleviate dropouts, and for
InterpGES runs also improves GSI but not enough
to alleviate dropouts meaning that adding the
SH observations (and the 3, 6, 9-h ges) degrade
the ECM forecasts.
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
SH Dropout Composite Cntrl vs InterpGES
SH CNTRL 5-day Composite AC 0.71
SH InterpGES 5-day Composite AC 0.73
Zonal average (above) and mean composite
difference between GFS Cntrl and InterpGES
(Increments). GFS has low level SH warm bias
if we consider ECMWF as ground truth.
26
Comparison of CNTRL vs. InterpGES Expt.
Guess
Analysis

• InterpGES run introduces a lower height and
  temperature bias to GSI
• ANL differences are not significantly different
  after GSI interpolation but CNTRL maintains high
  bias

Dropout Date GFS ECMWF ECM CNTRL INTERPGES
2008042600 0.61 0.91 0.89 0.65 0.72
27
Impact of GSI and OBSThe addition of GSI
Observations causes a positive height bias in the
SH
CNTRL
InterpGES
Typical Dropout case
Dropout Date GFS ECMWF ECM CNTRL INTERPGES
2008042600 0.61 0.91 0.89 0.65 0.72
28
EU Satwinds are not used in the GSI
The GSI draws more for most Satwinds
29
The GSI draws more for sonde winds especially
away from jet level
30
The GSI draws more for outlier winds less so
around jet level
31
ECM fits better 300 hPa and up
The draw for all temperatures is similar But
see next slide
32
The GSI draws more for outlier temperatures in
the middle atmosphere and the ECM draws more in
the jet level to upper atmosphere
33
Satellite Wind Speed Biases OB-ANL Apr 2008 12Z
ECM (Red), GSI (Blue) m/sec
Note ECM biases are mostly negative compared to
GSI, meaning ECM winds are stronger than GSI
winds at satellite wind locations
34
Baroclinicity parameter for GFS and ECMWF
A suitable measure of the baroclinicity is
given by the Eady growth rate maximum defined
as sBI 0.31 f (-gp/RT) ?V/ ?p N-1
where f is the Coriolis parameter, V is the total
vector wind, N is the Brunt Väisällä frequency
and all other parameters have their usual
meaning. This is done to show potential action
areas or volatility to propagate initial
condition errors into forecast differences
35
NH Eady Stability index Difference GFS - ECMWF
Largest differences for this NH Dropout case,
20081021, are in Pacific (disregard most
mountain areas as quantities are extrapolated
values below Ground).
SH Eady Stability index GFS - ECMWF
The predominance of Red over Blue shows GFS has
more pronounced baroclinicity compared to ECMWF
Ops in low levels (800mb)
36
NH GFS Eady Stability index
The total Eady-index shows the Greatest
baroclinic potential in the eastern part of the
broad Pacific trough, the differences in this
case, cause a dropout in the 5-day forecast.
NH ECMWF Eady Stability index
37
NH Eady Stability index GFS - ECMWF
Large differences are along trough line with
dipole, indicated by Blue and Red indicating
difference in position (phase), and large
potential for this Rossby wave details.
SH Eady Stability index GFS - ECMWF
And at 500 mb and 200 mb (not shown), GFS has
more pronounced baroclinicity compared to ECMWF
Ops. The differences are as much as 20 of the
total index shown on next slide.