Validation of Satellite Precipitation Estimates for Weather and Hydrological Applications

1
Validation of Satellite Precipitation Estimates
for Weather and Hydrological Applications

• Beth Ebert
• BMRC, Melbourne, Australia

3rd IPWG Workshop / 3rd APSATS, 23 October 2006,
Melbourne
2
Satellite precipitation estimates -- what do we
especially want to get right?
Climatologists - mean bias NWP data assimilation
(physical initialization) - rain location and
type Hydrologists - rain volume Forecasters and
emergency managers - rain location and maximum
intensity
3
Short-term precipitation estimates

• High spatial and temporal resolution desirable
• Dynamic range required
• Motion may be important for nowcasts
• Can live with some bias in the estimates if it's
  not too great
• Verification data need not be quite as accurate
  as for climate verification
• Land-based rainfall generally of greater interest
  than ocean-based

4
Some truths about "truth" data

• No existing measurement system adequately
  captures the high spatial and temporal
  variability of rainfall.
• Errors in validation data artificially inflate
  errors in satellite precipitation estimates

5
Rain gauge observations Advantages Disa
dvantages True rain measurements May be
unrepresentative of aerial
value Verification results biased
toward regions with high
gauge density Most obs made
once daily
6
Radar data Advantages Disadvantages
Excellent spatial and Beamfilling,
attenuation, temporal resolution overshoot,
clutter, etc. Limited spatial extent
7
Rain gauge analyses Advantages Disadvantages
Grid-scale quantities Smoothes actual rainfall
Overcomes uneven values distribution
of rain gauges
8
Verification strategy for satellite precipitation
estimates
Use (gauge-corrected) radar data for local
instantaneous or very short-term estimates Use
gauge or radar-gauge analysis for larger spatial
and/or temporal estimates
9
Spatial verification methods

• Visual ("eyeball") verification
• Continuous statistics (RMS, correlation, bias,
  etc)
• Categorical statistics (POD, FAR, etc.)
• - - - - - - - - - - - - - - - - - - - - - - - - -
  - - - - - - - - - - - - - - - - - - - - - - - - -
  - - - - - - - - - - - -
• Scale decomposition methods
• Entity-based methods

"standard"
"scientific" or "diagnostic"
10
Step 1 Visual ("eyeball") verification
Visually compare maps of satellite estimates and
observations Advantage "A picture tells a
thousand words" Disadvantages Labor intensive,
not quantitative, subjective
Verifies this attribute? Location Size Shape Mean
value Maximum value Spatial variability
Rozumalski, 2000
11
Time series of error statistics
24-hr rainfall from NRL Experimental
Geostationary algorithm validated against
Australian operational daily rain gauge
analysis 0.25 grid boxes, tropics only
12
Continuous statistics
Mean error (bias)
Measures average error
Mean absolute error
Measures average magnitude of error
Root mean square error
Measures error magnitude, with large errors
having a greater impact than in the MAE
Correlation coefficient
Measures correspondence between estimated spatial
distribution and observed spatial distribution,
independent of mean bias
13
Categorical statistics
14
Categorical statistics
Bias score
Probability of Detection
False Alarm Ratio
Threat score (critical success index)
Equitable threat score
15
Real-time verification example
24-hr rainfall from NRL Experimental
Geostationary algorithm
16
Web links for satellite precipitation validation

• IPWG validation of daily rainfall estimates over
  many regions - http//www.bom.gov.au/bmrc/SatRainV
  al/validation-intercomparison.html
• Climate Rainfall Data Center (monthly validation,
  global) - http//rain.atmos.colostate.edu/CRDC/
• Program for the Evaluation of High Resolution
  Precipitation Products - http//essic.umd.edu/7Em
  sapiano/PEHRPP/
• Validation of 6-hourly and daily Hydro-Estimator
  and other geostationary estimates over US -
  http//www.orbit.nesdis.noaa.gov/smcd/emb/ff/valid
  ation/validation.html
• TOVAS validation of daily TRMM-based estimates
  over US - http//disc2.nascom.nasa.gov/Giovanni/to
  vas/rain.ipwg.shtml
• Forecast verification Issues, Methods, and FAQ
  - http//www.bom.gov.au/bmrc/wefor/staff/eee/verif
  /verif_web_page.html