Long-term trends of precipitation verification results for GME, COSMO-EU and COSMO-DE

1
Long-term trends of precipitation verification
results for GME, COSMO-EU and COSMO-DE

• Ulrich Damrath

2
Overview

• Verification using Fuzzy methods
• Example for the FSS and he upscaling HSS for
  January and July
• Time series for special thresholds and window
  sizes
• Consisency of precipitation forecasts using the
  CRA method (update)
• current state

3
Fuzzy verification January 2008 Accumulated
frequency distribution of precipitation,
observation and GME
4
Fuzzy verification January 2008 Accumulated
frequency distribution of precipitation,
observation and CEU
5
Fuzzy verification January 2008 Accumulated
frequency distribution of precipitation,
observation and CDE
6
Fuzzy verification January 2008 FSS
Monthly average of precipitation 68 mm
7
Fuzzy verification January 2009 FSS
Monthly average of precipitation 30 mm
8
Fuzzy verification January 2010Frequency
distribution of precipitationObservation, GME,
CEU and CDE
9
Fuzzy verification January 2010 FSS
Monthly average of precipitation 44 mm
10
Accumulated frequency distribution of
precipitationJuly 2007 (Observation and GME)
11
Accumulated frequency distribution of
precipitationJuly 2007 (Observation and CEU)
12
Accumulated frequency distribution of
precipitationJuly 2007 (Observation and CDE)
13
Fuzzy verification July 2007 FSS
Monthly average of precipitation 120 mm
14
Fuzzy verification July 2008 FSS
Monthly average of precipitation 88 mm
15
Fuzzy verification July 2009 FSS
Monthly average of precipitation 108 mm
16
Fuzzy verification July 2010Frequency
distribution of precipitationObservation, GME,
CEU and CDE
17
Fuzzy verification July 2010 FSS
Monthly average of precipitation 78 mm
18
Fuzzy verification January 2008 HSS(UPS)
Monthly average of precipitation 68 mm
19
Fuzzy verification January 2009 HSS(UPS)
Monthly average of precipitation 30 mm
20
Fuzzy verification January 2010Frequency
distribution of precipitationObservation, GME,
CEU and CDE
21
Fuzzy verification January 2010 HSS(UPS)
Monthly average of precipitation 44 mm
22
Fuzzy verification July 2007 HSS(UPS)
Monthly average of precipitation 120 mm
23
Fuzzy verification July 2008 HSS(UPS)
Monthly average of precipitation 88 mm
24
Fuzzy verification July 2009 HSS(UPS)
Monthly average of precipitation 108 mm
25
Fuzzy verification July 2010Frequency
distribution of precipitationObservation, GME,
CEU and CDE
26
Fuzzy verification July 2010 HSS(UPS)
Monthly average of precipitation 78 mm
27
Fuzzy verification Time series, choice of
windows and thresholds
28
Fuzzy verification Time series, ETS UPS GME
VV06-30
29
Fuzzy verification Time series, ETS UPS CEU
VV06-30
30
Fuzzy verification Time series, ETS UPS GME
VV06-18
31
Fuzzy verification Time series, ETS UPS CEU
VV06-18
32
Fuzzy verification Time series, ETS UPS CDE
VV06-18
33
Fuzzy verification Time series, FSS GME VV06-30
34
Fuzzy verification Time series, FSS CEU VV06-30
35
Fuzzy verification Time series, FSS GME VV06-18
36
Fuzzy verification Time series, FSS CEU VV06-18
37
Fuzzy verification Time series, FSS CDE VV06-18
38

• Entity-based QPF verification (rain blobs)
• by E. Ebert (BOM Melbourne)
• Verify the properties of the forecast rain system
  against the properties of the observed rain
  system
• location
• rain area
• rain intensity (mean, maximum)

CRA error decomposition The total mean squared
error (MSE) can be written as MSEtotal
MSEdisplacement MSEvolume MSEpattern
Configuration for the current study -
Observations forecasts 06-30 hours -
Forecasts forecasts 30-54 hours and
forecasts 54-78 hours
39
Consistency of precipitation forecasts Parts of
error decomposition Autumn 2009
Dark forecasts 30-54 h Lightforecasts 54-78 h
40
Consistency of precipitation forecasts Parts of
error decomposition Winter 2009/10
Dark forecasts 30-54 h Lightforecasts 54-78 h
41
Consistency of precipitation forecasts Parts of
error decomposition Spring 2010
Dark forecasts 30-54 h Lightforecasts 54-78 h
42
Consistency of precipitation forecasts Parts of
error decomposition Summer 2010
Dark forecasts 30-54 h Lightforecasts 54-78 h
43
Consistency of precipitation forecasts Parts of
error decomposition Summer 2010
Dark forecasts 30-54 h Lightforecasts 54-78 h
44
Summary

• Fraction skill score and upscaling ETS are
  considered. Both scores are relatively high
  correlated.
• Fuzzy verification in general shows best results
  for low precipitation values and large window
  sizes
• For some months best results can be seen for
  precipitation amounts around 2 mm (12 h)-1
• CEU and CDE models have nearly the same quality
  and are better than GME especially during summer
  times.
• A positive long term trend of precipitation
  quality can be seen for low precipitation values
  and large window sizes. No clear trend is visible
  for high precipitation values for any window
  size.
• Results for the check of consistency of
  precipitation forecasts lead to the expected (but
  proved) results that for high thresholds the
  inconsistency is most obvious. During winter time
  pattern errors are dominant. During summer times
  displacement errors are prevailing.

45
One conclusion

• Forecasters sometimes really like CDE.
• But the future is CDE-EPS!