Title: Findings and Recommendations from the Hydrologic Verification System Requirements Team
1Findings and Recommendations from the Hydrologic
Verification System Requirements Team
- Peter Gabrielsen, Julie Demargne, Mary Mullusky,
Kevin Werner, Bill Lawrence, Scott Lindsey, Donna
Page, Noreen Schwein, Scott Staggs, Tom Adams,
and Bill Marosi
RFC Verification Workshop, August 14-16 2007
2Final Reportavailable at
http//www.nws.noaa.gov/oh/rfcdev/docs/Final_Verif
ication_Report.pdf
3Outline
- Forecast Verification at NWS needs and vision
- River Forecast Verification System
- Recommendations vs. Current/Future Work
- Resources for Verification
- Verification Plan
- Verification Activities
4Forecast Verification at NWS
- Needs
- 1996 National Research Council stated
verification of hydrologic forecasts was
inadequate - 2006 Board on Atmospheric Sciences and Climate
recommended NWS to expand verification of its
uncertainty products and make it easily available
to all users in near real time - Vision
- Develop a comprehensive national system to verify
hydrologic forecasts and guidance products which
satisfy the needs of all users - Improve forecast services by analyzing sources of
error and skill across the entire forecast
process - Provide easy access to enhanced river forecast
verification data to improve our scientific and
operational techniques and services
Completing the Forecast, Characterizing and
Communicating Uncertainty for Better Decisions
Using Weather and Climate Forecasts
5River Forecast Verification System
- Goals
- Quantify quality of RFC forecasts and quality of
forecast services - Monitor forecast quality over time
- Monitor quality at various steps in the forecast
process - Improve forecast quality
- Assist prioritization of forecast system
enhancements - Uses
- Operational
- Experimental/Research
- Customers
- Hydrologic forecasters
- Scientists/Researchers
- Hydrologic program managers
- Emergency and water resources managers
- Public
6River Forecast Verification System
- Components
- Logistical verification
- Deterministic verification
- Probabilistic verification
- Verification System Capabilities
- Data Archiving (attributes for time, service,
basin and events) - Computing Metrics
- Displaying Metrics (graphics and reports)
- Disseminating Metrics and Data (understand
quality usefulness of forecasts) - Real Time Access to Metrics (understand errors in
recent forecasts and over long term) - Error Analysis (including hindcast experiments)
- Performance Measure Tracking (level of success)
7Verification Resources
- Recommended resources
- OHD Program Manager for verification
- RFC Verification Focal Points at each RFC
- National Team for developing and implementing
standardized formats and procedures to archive
data required for verification - Current and near-future resources
- National Verification Focal Points Julie
Demargne Mary Mullusky support from others - 13 RFC Verification Focal Points
- 2 Hydrologic Forecast Verification Teams for WR
and NWS
8Proposed Verification Plan
- Recommendations from FY07 up to FY11
Probabilistic Verification
Deterministic Verification
Task
1. Archive
2. Compute Metrics
3. Display Metrics
4. Dissemination
5. Real Time Access
6. Error Analysis
7. Performance Tracking
Logistical Verification
Grid Verification
Task
1. Archive
2. Compute Metrics
3. Display Metrics
4. Dissemination
5. Real Time Access
6. Error Analysis
7. Performance Tracking
9AHPS FY07 Proposed Verification Activities
- Deterministic Verification
- Enhance release IVP
- Develop error analysis, raw model forecasts,
deterministic hindcaster - Define data requirements hardware for
verification hindcaster - Ensemble Verification
- Enhance release EVS
- Enhance release Ensemble Hindcaster Develop
error analysis - Evaluate existing archiving capabilities
- Logistical Verification
- Develop plan for archiving info
- Implement logistical measures
10Current Verification Activities
- Deterministic Verification
- Deterministic Verification Tools IVP
enhancements release - Develop Error Analysis development of raw model
forecast application deterministic hindcaster,
define data requirements hardware - Ensemble Verification
- Improve Ensemble Verification EVS enhancements
release, development of requirements for
archiving capabilities - Improve Ensemble Hindcaster EH enhancements
release, development of uncertainty analysis
capabilities - Logistical Verification
- Develop plan for archiving info
- Implement logistical measures
11Future Verification Activities
- Verification System Requirements Planning Team
to define the AHPS FY08 activities and update
verification plan - AHPS activities to be continued
- Deterministic Verification IVP, Error Analysis
- Ensemble Verification EVS, Error Analysis
- Logistical Verification Logistical Measures
- 2 Hydrologic Forecast Verification Teams for WR
and NWS to develop verification strategies using
case studies
12Thank you