Code Optimization

1
An Improved Wind Probability Program A Year 2
Joint Hurricane Testbed Project Update
Mark DeMaria and John Knaff, NOAA/NESDIS, Fort
Collins, CO Stan Kidder, CIRA/CSU, Fort Collins,
CO Buck Sampson, NRL, Monterey, CA Chris Lauer
and Chris Sisko, NCEP/TPC, Miami, FL
Presented at the Interdepartmental Hurricane
Conference March 5, 2009
2
Monte Carlo Wind Probability Model

• Estimates probability of 34, 50 and 64 kt wind to
  5 days
• Implemented at NHC for 2006 hurricane season
• Replaced Hurricane Strike Probabilities
• 1000 track realizations from random sampling NHC
  track error distributions
• Intensity of realizations from random sampling
  NHC intensity error distributions
• Special treatment near land
• Wind radii of realizations from radii CLIPER
  model and its radii error distributions
• Serial correlation of errors included
• Probability at a point from counting number of
  realizations passing within the wind radii of
  interest

3
MC Probability Example Hurricane Ike 7 Sept 2008
12 UTC
1000 Track Realizations 64 kt
0-120 h Cumulative Probabilities
4
Project Tasks

• Improved Monte Carlo wind probability program by
  using situation-depending track error
  distributions
• Track error depends on Goerss Predicted Consensus
  Error (GPCE)
• Improve timeliness by optimization of MC code
• Update NHC wind speed probability product
• Extend from 3 to 5 days
• Update probability distributions (was based on
  1988-1997)

5
Tasks 2 and 3 Completed

• Optimized code implemented for 2007 season
• Factor of 6 speed up
• Wind Speed Probability Table
• Calculated directly from MC model intensity
  realizations
• Implemented for 2008 season

6
Task 1 Forecast Dependent Track Errors

• Use GPCE input as a measure of track uncertainty
• Divide NHC track errors into three groups based
  on GPCE values
• Low, Medium and High
• For real time runs, use probability distribution
  for real time GPCE value tercile
• Different forecast times can use different
  distributions
• Relies on relationship between NHC track errors
  and GPCE value

7
Goerss Predicted Consensus Error (GPCE)

• Predicts error of CONU track forecast
• Consensus of GFDI, AVNI, NGPI, UKMI, GFNI
• GPCE Input
• Spread of CONU member track forecasts
• Initial latitude
• Initial and forecasted intensity
• Explains 15-50 of CONU track error variance
• GPCE estimates radius that contains 70 of CONU
  verifying positions at each time
• In 2008, GPCE predicts TVCN error
• GFS, UKMET, NOGAPS, GFDL, HWRF, GFDN, ECMWF

8
72 hr Atlantic NHC Along Track Error
Distributions Stratified by GPCE
9
2008 Evaluation Procedure

• GPCE version not ready for 2008 real time
  parallel runs
• Re-run operational and GPCE versions for 169
  Atlantic cases within 1000 km of land at t0
• Qualitative Evaluation Post 34, 50, 64 kt
  probabilities on web page for NHC
• Operational, GPCE and difference plots
• Quantitative Evaluation Calculate probabilistic
  forecast metrics from output on NHC breakpoints

10
GPCE MC Model Evaluation Web Page
http//rammb.cira.colostate.edu/research/tropical_
cyclones/tc_wind_prob/gpce.asp
11
Individual Forecast Case Page
12
Tropical Storm Hanna 5 Sept 2008 12 UTC
34 kt 0-120 h cumulative probability difference
field (GPCE-Operational) All GPCE values in
High tercile
13
Hurricane Gustav 30 Aug 2008 18 UTC
64 kt 0-120 h cumulative probability difference
field (GPCE-Operational) All GPCE values in Low
tercile
14
Quantitative Evaluation

• Calculate probabilities at NHC breakpoints
• Operational and GPCE versions
• 34, 50 and 64 kt
• 12 hr cumulative and incremental to 120 h
• 169 forecasts X 257 breakpoints 43,433 data
  points at each forecast time
• Two evaluation metrics
• Brier Score
• Optimal Threat Score

15
Operational and GPCE Probabilities Calculated at
257 NHC Breakpoints
West Coast of Mexico and Hawaii
breakpoints excluded to eliminate zero or very
low probability points
16
Brier Score (BS)

• Common metric for probabilistic forecasts
• Pi MC model probability at a grid point (0 to
  1)
• Oi Observed probability (1 if yes, 0 if no)
• Perfect BS 0, Worst 1
• Calculate BS for GPCE and operational versions
• Skill of GPCE is percent improvement of BS

17
Brier Score Improvements2008 GPCE MC Model Test
Cumulative
Incremental
18
Threat Score (TS)

• Choose a probability threshold to divide between
  yes or no forecast
• Calculate Threat Score (TS)
• Repeat for wide range of thresholds
• Every 0.5 from 0 to 100
• Find maximum TS possible
• Compare best TS for GPCE and operational model
  runs

a
c
b
Observed Area
Forecast Area
19
Threat Score Improvements2008 GPCE MC Model Test
Cumulative
Incremental
20
Potential Impact of GPCE on Hurricane Warnings

• Automated hurricane warning guidance from MC
  probabilities under development
• Schumacher et al. (2009 IHC)
• Warning algorithm run for Hurricane Gustav (2008)
• Operational and GPCE versions

21
Summary

• Code optimization and wind speed table product
  are complete
• Implemented before 2007 and 2008 seasons
• GPCE-dependent MC model
• Tested on 169 Atlantic cases from 2008
• Results are qualitatively reasonable
• Improves Brier Score at all time periods relative
  to operational MC model
• Improves Threat Score at most time periods
• Not tested in the eastern and western Pacific