Development and Calibration of Ensemble Based Hazardous Weather Products at the Storm Prediction Cen

1
Development and Calibration of Ensemble Based
Hazardous Weather Products at the Storm
Prediction Center

• David Bright
• Gregg Grosshans, Jack Kain, Jason Levit, Russ
  Schneider, Dave Stensrud, Matt Wandishin, Steve
  Weiss
• October 11, 2005
• NCEP Predictability Discussion Group

Where Americas Climate and Weather Services Begin
2
STORM PREDICTION CENTER
MISSION STATEMENT The Storm Prediction Center
(SPC) exists solely to protect life and property
of the American people through the issuance of
timely, accurate watch and forecast products
dealing with hazardous mesoscale weather
phenomena.

• MISSION STATEMENT
• The Storm Prediction Center (SPC) exists
• solely to protect life and property of the
  American people
• through the issuance of timely, accurate watch
  and forecast products
• dealing with tornadoes, wildfires and other
  hazardous mesoscale weather phenomena.

3
STORM PREDICTION CENTER
HAZARDOUS PHENOMENA

• Hail, Wind, Tornadoes
• Excessive rainfall
• Fire weather
• Winter weather

4
SPC Forecast Products

• TORNADO SEVERE THUNDERSTORM WATCHES
• WATCH STATUS MESSAGE
• CONVECTIVE OUTLOOK
• Day 1 Day 2 Day 3 Days 4-8
• MESOSCALE DISCUSSION
• Severe Thunderstorm Potential/Outlook Upgrade
• Thunderstorms not expected to become severe
• Hazardous Winter Weather
• Heavy Rainfall
• FIRE WEATHER OUTLOOK
• Day 1 Day 2 Days 3-8
• OPERATIONAL FORECASTS ARE BOTH DETERMINISTIC AND
  PROBABILISTIC

75 of all SPC products are valid for lt 24h
period
5
EXPERIMENTAL WATCH PROBABILITIES
Severe Thunderstorm Watch 688 Probability Table
6
CONVECTIVE OUTLOOKSOperational through Day 3
7
Thunderstorm Outlooks
12h Periods (gt 10 40 70)
12h Enhanced Thunderstorm (Today)
24h General Thunderstorm
24h Period (gt 10)
12h Enhanced Thunderstorm (Tonight)
8
Product Guidance at the SPC

• Operational emphasis on
• Observational data
• Short-term, high-resolution NWP guidance
• Specific information predicting hazardous
  mesoscale phenomena
• NWP needs range from the very-short range to
  medium range
• Very short-range Hourly RUC 4.5 km WRF-NMM
• Short-range NAM, GFS, SREF
• Medium-range GFS, ECMWF, MREF
• Todays focus SREF
• Overview of the ensemble product suite
• Specific ensemble calibrated guidance

9
Objective Provide a wide range of ensemble
guidance covering all of the SPC program areas
Overview of Ensemble Guidance
10
Sample of Ensemble Products Available
MEAN SD 500 mb HGHT
SPAGHETTI SFC LOW
MEAN MUCAPE, 0-6 SHR, 0-3 HLCY
MEAN PMSL, DZ, 10M WIND
http//www.spc.noaa.gov/exper/sref/
11
Sample of Ensemble Products Available
STP F (mlCAPE, mlLCL, SRH, Shear) Thompson et
al. (2003)
Omega lt -3 -11 lt T lt -17 RH gt 80
PROB DENDRITIC GROWTH
PROB SIG TOR PARAM gt 3
STP F (mlCAPE, mlLCL, SRH, Shear) Thompson et
al. (2003)
MEDIAN, UNION, INTERSECTION SIG TOR PARAM
MAX OR MIN MAX FOSBERG
INDEX
http//www.spc.noaa.gov/exper/sref/
12
F63 SREF POSTAGE STAMP VIEW PMSL, HURRICANE
FRANCES
Red EtaBMJ Yellow EtaKF Blue
RSM White OpEta
SREF Member
13
Combined Probability
CAPE (J/kg) Green solid Percent Members gt 1000
J/kg Shading gt 50 Gold dashed Ensemble mean
(1000 J/kg) F036 Valid 21 UTC 28 May 2003

• Probability surface CAPE gt 1000 J/kg
• Relatively low
• Ensemble mean is lt 1000 J/kg (no gold
  dashed line)

14
Combined Probability
10 m 6 km Shear (kts) Green solid Percent
Members gt 30 kts Shading gt 50 Gold dashed
Ensemble mean (30 kts) F036 Valid 21 UTC 28 May
2003

• Probability deep layer shear gt 30 kts
• Strong mid level jet through Iowa

15
Combined Probability
3 Hour Convective Precipitation gt 0.01
(in) Green solid Percent Members gt 0.01 in
Shading gt 50 Gold dashed Ensemble mean (0.01
in) F036 Valid 21 UTC 28 May 2003

• Convection is likely WI/IL/IN
• Will the convection become severe?

16
Combined Probability
Prob Cape gt 1000 X Prob Shear gt 30 kts
X Prob Conv Pcpn gt .01

F036 Valid 21 UTC 28 May 2003

• A quick way to determine juxtaposition of key
  parameters
• Fosters an ingredients-based approach
• Not a true probability
• Dependence
• Different members contribute

17
Combined Probability
Prob Cape gt 1000 X Prob Shear gt 30 kts
X Prob Conv Pcpn gt .01

F036 Valid 21 UTC 28 May 2003

• A quick way to determine juxtaposition of key
  parameters
• Fosters an ingredients-based approach
• Not a true probability
• Dependence
• Different members contribute

Severe Reports RedTor BlueWind GreenHail
18
Combined Probability
Ingredients for extreme fire weather conditions
over the Great Basin
F15 SREF PROBABILITY TPCP x RH x WIND x TMPF (lt
.01 x lt 10 x gt 30 mph x gt 60 F)
19
Objective Develop calibrated probabilistic
guidance for CG lightning
Calibrated Thunderstorm Guidance
20
Combine Lightning Ingredients into a Single
Parameter

• Three first-order ingredients (readily available
  from NWP models)
• Lifting condensation level gt -10o C
• Sufficient CAPE in the 0o to -20o C layer
• Equilibrium level temperature lt -20o C
• Cloud Physics Thunder Parameter (CPTP)
• CPTP (-19oC Tel)(CAPE-20 K)
• K
• where K 100 Jkg-1 and CAPE-20 is MUCAPE in the
  
• 0o C to -20o C layer

21
Example CPTP One Member
18h Eta Forecast Valid 03 UTC 4 June 2003
Plan view chart showing where grid point
soundings support lightning (given a convective
updraft)
22
SREF Probability CPTP gt 1
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncalibrated probability Solid/Filled Mean
CPTP 1 (Thick dashed)
23
SREF Probability Precip gt .01
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncalibrated probability Solid/Filled Mean
precip 0.01 (Thick dashed)
24
Joint Probability (Assume Independent)
P(CPTP gt 1) x P(Precip gt .01) 3 hr valid period
21 UTC 31 Aug to 00 UTC 01 Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncalibrated probability Solid/Filled
25
Uncalibrated Reliability (5 Aug to 5 Nov 2004)
Frequency 0, 5, , 100
Perfect Forecast
No Skill
Climatology
P(CPTP gt 1) x P(P03I gt .01)
26
Adjusting Probabilities

• Calibrate ensemble thunderstorm guidance based on
  the observed frequency of occurrence

27
Ensemble Thunder Calibration

• Bin separately P(CPTP gt 1) and P(P03M gt 0.01)
  into 11 bins (0-5 5-15 85-95 95-100)
• Combine the two binned probabilistic forecasts
  into one of 121 possible combinations (0,0)
  (0,10) (100,100)
• Use NLDN CG data over the previous 366 days to
  calculate the frequency of occurrence of CG
  strikes for each of the 121 binned combinations
• Construct for each grid point using 1/r weighting
• Bin ensemble forecasts as described in steps 1
  and 2 and assign the observed CG frequency (step
  3) as the calibrated probability of a CG strike
• Calibration is performed for each forecast cycle
  (09 and 21 UTC) and each forecast hour domain is
  entire U.S. on 40 km grid (CG strike within 12
  miles)

28
Before Calibration
29
Joint Probability (Assumed Independence)
P(CPTP gt 1) x P(Precip gt .01) 3 hr valid period
21 UTC 31 Aug to 00 UTC 01 Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Uncorrected probability Solid/Filled
30
After Calibration
31
Calibrated Ensemble Thunder Probability
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Calibrated probability Solid/Filled
32
Calibrated Ensemble Thunder Probability
3 hr valid period 21 UTC 31 Aug to 00 UTC 01
Sept 2004
15h Forecast Ending 00 UTC 01 Sept
2004 Calibrated probability Solid/Filled NLDN
CG Strikes (Yellow )
33
Calibrated Reliability (5 Aug to 5 Nov 2004)
Frequency 0, 5, , 100
Perfect Forecast
Perfect Forecast
No Skill
Climatology
No Skill
Calibrated Thunder Probability
34
3h probability of gt 1 CG lightning strike within
12 mi 09Z and 21Z SREF valid at F003 through
F063 May 15 Sept 15 2005
Economic Potential Value
Reliability
35
12h probability of gt 1 CG lightning strike within
12 mi 09Z SREF valid at F012 through F063 May
15 Sept 15 2005
Economic Potential Value
Reliability
36
Objective Develop calibrated probabilistic
guidance of the occurrence of severe convective
weather(Available for 3h, 12h, and 24h periods
calibration not described today)
Calibrated Severe Thunderstorm Guidance
37
24h probability of gt 1 severe thunderstorm within
25 mi SREF 2005051109 Valid 12 UTC May 11, 2005
to 12 UTC May 12, 2005
SVR WX ACTIVITY 12Z 11 May to 12Z 12 May, 2005 a
Hail wWind tTornado
38
24h probability of gt 1 severe thunderstorm within
25 mi 21Z SREF valid at F039 through F039 (i.e.,
Day 1 Outlook) May 15 Sept 15 2005
Economic Potential Value
Reliability
Hail gt .75
Wind gt 50 kts
Tornado
39
Objective Develop calibrated probabilistic
guidance of snow accumulation on road surfaces
Experimental Calibrated Snow Accumulation Guidance
40
Ensemble Snow Calibration

• Use frequency of occurrence technique -- similar
  to the calibrated probability of CG lightning
• Produce 8 calibrated joint probability tables
• Take power mean (RMS average) of all 8 tables for
  the 3h probability of snow accumulating on roads
  in the grid cell
• Calibration period is Oct. 1, 2004 through Apr.
  30, 2005
• MADIS road-state sensor information is truth
  (SREF is interpolated to MADIS road sensor)

41
Goal Examine the parameter space around the
lower PBL T, ground T, and precip type and
calibrate using road sensor data.

• SREF probability predictors
• Two precipitation-type algorithms
• Baldwin algorithm in NCEP post. (PrSn, ZR, IP)
• Czys algorithm applied in SPC SREF
  post-processing.
• (PrSn, ZR, IP)
• (2) Two parameters sensitive to lower
  tropospheric and ground temperature
• Snowmelt parameterization (RSAE) Evaluates
  fluxes to determine if 3 of snow melts over a 3h
  period. If yes, then parameter is assigned
  273.15 TG. (Prgt1 gt2 gt4)
• Simple algorithm (RSAP) F (Tpbl, TG, Qsfc net
  rad. flux,) where values gt 1 indicate surface
  cold enough for snow to accumulate. (Prgt1)

42
Frequency Calibration Tables

• LAYER SREF INGREDIENT 1
  SREF INGREDIENT 2
  1 Prob(RSAE gt 1)
  Prob(Baldwin Snow, ZR, or IP) 2
  Prob(RSAE gt 2) Prob(Baldwin Snow,
  ZR, or IP)
• 3 Prob(RSAE gt 4)
  Prob(Baldwin Snow, ZR, or IP)
• 4 Prob(RSAE gt 1)
  Prob(Czys Snow, ZR, or IP) 5
  Prob(RSAE gt 2) Prob(Czys Snow, ZR, or IP)
• 6 Prob(RSAE gt 4)
  Prob(Czys Snow, ZR, or IP)
• 7 Prob(RSAP gt 1)
  Prob(Baldwin Snow, ZR, or IP)
• 8 Prob(RSAP gt 1)
  Prob(Czys Snow, ZR, or IP)

43
Example New England Blizzard (F42 23 January
2005 03Z)
SREF 32F Isotherm (2 meter air temp) Mean
(dash) Union (At least one SREF member at or
below 32 F - dots) Intersection (All members at
or below 32F- solid)
SREF 32F Isotherm (Ground Temp) Mean
(dash) Union (At least one SREF member at or
below 32 F - dots) Intersection (All members at
or below 32F- solid)
3h probability of freezing or frozen pcpn
(Baldwin algorithm uncalibrated)
3h calibrated probability of snow accumulating on
roads
44
Example Washington, DC Area (F21 28 February
2005 18Z)
SREF 32F Isotherm (2 meter air temp) Mean
(dash) Union (dots) Intersection (solid)
SREF 32F Isotherm (Ground Temp) Mean
(dash) Union (dots) Intersection (solid)
3h probability of freezing or frozen pcpn
(Baldwin algorithm uncalibrated)
3h calibrated probability of snow accumulating on
roads
45
6h Prob Snow Accum on Roads Oct 15, 2005 (F006
v15 UTC)
3h Prob Snow Accum on Roads Oct 15, 2005 (F006
v15 UTC)
46
Blind Test

• Calibration period Oct 1, 2004 through April 30,
  2005
• 5 days randomly selected for each month in the
  sample gt 35 days in test
• Test days withheld from the monthly calibration
  tables (i.e., cross validation used)
• The SREF forecasts were reprocessed for the 35
  days and verified against the MADIS surface state
  observations (F03 F63)

47
Verification
Economic Potential Value
Reliability
Reliability Diagram All 3 h forecasts (F00
F63) 35 days (Oct 1 Apr 30)
48
Test Results

• 3 h forecast results (F00 F63)
• Forecast are reliable
• Brier score is a 21 improvement over sample
  climatology
• ROC area .919
• Ave probability where new snow detected 23
• Ave probability where new snow not detected 4
• Economic value for a wide range of users peaking
  over 0.7

49
Road-Snow Summary

• Method appears reliable although 3h
  probabilities rarely exceed 50
• Highlights importance of ground temp predictions
  from SREF and deterministic models
• Possible improvements
• Bias correction to 2m and ground temps from SREF
• Statistical post-processing of 2m and ground
  temps prior to road-state calibration
• Addition of asphalt tile to LSM of SREF members

See the next slide for temp correction
information
50
Under dispersive SREF 2m temp forecast (F15)
and cold bias
Raw 2m Temp
F15 SREF 2m Temp Verf Period August, 2005
F15 cold bias in 2m temp removed but remains
under dispersive
Bias adjusted 2m Temp
Bias adjustment and recalibration with the
addition of asphalt-type ground temp tile in LSM
might be very useful for snow accumulation from
SREF
Recalibrated 2m Temp
Uniform VOR after statistical adjustment to SREF