Huug van den Dool and Suranjana Saha Prediction Skill and Predictability in CFS

1
Huug van den Dool and Suranjana Saha
Prediction Skill and Predictability in CFS
2
Definitions Prediction Skill and
PredictabilityOpinion Literature fuzzies up
predictability vs prediction skill
3
Definition 1 Evaluation of skill of real time
prediction the old-fashioned way. Problems a)
Sample size! , b) Wait a long time(and funding
agents are impatient)
4
Definition 1 Evaluation of skill of real time
prediction the old-fashioned way. Definition
2 Evaluation of skill of hindcasts hard, not
impossible.Problems a) Sample size, b)
honesty of hindcasts
5
Definition 1 Evaluation of skill of real time
prediction the old-fashioned way. Definition
2 Evaluation of skill of hindcasts hard, not
impossible.Definition 3 Predictability of the
1st kind ( sensitivity due to uncertainty in
initial conditions)
6
Definition 1 Evaluation of skill of real time
prediction the old-fashioned way. Sample
size!Definition 2 Evaluation of skill of
hindcasts hard, not impossibleDefinition 3
Predictability of the 1st kind ( sensitivity due
to uncertainty in initial conditions)Definition
4 Predictability of the 2nd kind due to
variations in external boundary conditions (AMIP
Potential Predictability Reproducibility
Maddens approach)
7
Predictability (theoretical/intrinsic) is a
ceiling for actual prediction skill.Any other
kinds of predictability?
8
CFS forecastX (space, lead, member ,year)

• Space is 2.5oX2.5o (Z500) or 1oX2o (SST/mask), or
  1.875 by Gaussian (Soilw, T2m, Precip)
• Basic data used is monthly mean
• Lead 0, 8 in units of months member 1, 15
• Year 1981 2003 (increases annually)
• Example Initial Month is August ( lead 0)
• Note IC is Jul 11/21/Aug 1 for SST, and Jul
  09-13/ 19-23 / Jul 30-Aug3 for atmosphere and
  soil.
• Member 16 is ensemble average
• Member 17 is matching observed field
• X ( Z500, SST, Soilw, T2m, Precip)

9
ASPECTS

• Prediction skill (member i vs member 17)
• Predictability (member i vs member j)
• Monthly mean
• Seasonal mean
• Ensemble average
• Predictability of 1st kind only.

10
Two types of climatology plus complications

• Xclim_mdl (space, lead) is average over years and
  (14 or 15) members, depending.
• Xclim_verif (space, lead) is ave over (same)
  years for either member 17, or member i, i15.
• Anomaly X minus Xclim, whichever is relevant
• Systematic error (SE) is automatically corrected
  by the above
• CV of the SE correction (exclude from Xclim the
  member and the year to be verified). Not trivial.

11
Prediction Skill Monthly
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
Conclusions (monthly data)

• CFS data is a goldmine.
• CFS has enough (?) data for forecast evaluation
  (and diagnostics)
• Member i vs member j unifies predictability of
  1st and 2nd kind in CFS output
• CFS has some prediction skill. In order of skill
  SST, tropical variables, soilw,T2m, Precip
• CFS has some more predictability (as defined),
  but ceiling is low in mid-latitudes.
• Seasonality (no surprise)

30
To do

• Identify interdecadal skill source (if any)
• Identify soil moisture skill source (are models
  still too strong on local effects? How about
  non-local effects)
• Daily data for the finer temporal scales in
  skill/predictability.
• Why do models like CFS have predictability in so
  few d.o.f. (and is that really all there is)
• Further ideas about new predictability notions

31
A case for the importance of knowing the
effective number of degrees of freedom (edof) in
which we have forecast skill.Considerations-)
physical models have one clear strength they can
execute the non-linear terms-) a model needs at
least 3 degrees of freedom to be non-linear
(Lorenz, 1960)-) a non-linear model with
nominally a zillion degrees of freedom, but skill
in only lt 3 dof is functionally linear in terms
of the skill of its forecasts - and, to its
detriment, the non-linear terms add random
numbers to the tendencies of the modes with
predictability. gt Therefore Physical models
need to have skill in, effectively, gt 3 dof
before they can be expected to take advantage of
non-linearity. (In a forecast setting). (
Note not any 3 degrees of freedom will do.)
32
Lingering memory CaiVan den Dool(2005) Schemm
et al calibration data set, (CFS daily data set
will be used also).