Title: Progress on Radar Data Assimilation at the NCEP Environmental Modeling Center
1Progress on Radar Data Assimilationat the NCEP
Environmental Modeling Center
- S. Lord, G. DiMego, D. Parrish,
- NSSL Staff
- With contributions by
- J. Alpert, V. K. Kumar, R. Saffle, Q. Liu
NCEP where Americas climate, weather, and
ocean services begin
2Overview
- Introductory remarks
- NEXRAD observations and Data Assimilation (DA)
- History of NEXRAD data use in DA, including
precipitation assimilation (Lin, Parrish) - CONUS impact study (Alpert)
- Hurricane impact study (Liu)
- Summary and outlook
3NEXRAD WSR-88D RADARS
- 158 operational NEXRAD Doppler radar systems
deployed throughout the United States - Provide warnings on dangerous weather and its
location - Potentially useful for mesoscale data
assimilation - Data resolution of Level 2 radar radial wind
- 1/4 km radial resolution
- 1 degree of azimuth
- 16 vertical tilt angles
- 200 km range
- 8 minutes time resolution
- Wind observation processing
- VAD cartesian (u,v) wind from radial wind
processing - Level 3 dealiased radial wind at 4 lowest tilts
- Level 2.5 on-site processing by NCEP superob
algorithm - Level 2.0 raw radial wind
- Data volume
- 100 Billion (1011) potential reports/day for
radar radial winds - Typically 2 Billion radial wind reports/day
- 0.1 Tb/day computer storage
4NEXRAD WSR-88D RADARS
- A rich source of high resolution observations
- Radial (Line of Sight) wind
- Reflectivity ? precipitation
5NEXRAD WSR-88D RADARSLevel 2.5 Data Coverage
6Radar or Model Reflectivity?
WRF 24 hour 4.5 km forecast of 1 hour accumulated
precipitation valid at 00Z April 21, 2004 and
corresponding radar reflectivity
7Five Order of Magnitude Increase in Satellite
Data Over Next Ten Years
NPOESS Era Data Volume
Daily Satellite Radar Observation Count
2005 210 M obs
2003-4 125 M obs
Level 2 radar data 2 B
2002 100 M obs
Count (Millions)
2000
1990
2010
2010-10of obs
8Integration and Testing of New Observations
- Data Access (routine, real time) 3 months
- Formatting and establishing operational data
base 1 month - Extraction from data base 1 month
- Analysis development (I) 6-18
months - Preliminary evaluation 2 months
- Quality control 3 months
- Analysis development (II) 6-18
months - Assimilation testing and forecast evaluation 1
month - Operational implementation 6 months
- Maintain system 1 person till death do
us part
Total Effort 29-53 person months per instrument
Scientific improvements, monitoring and quality
assurance
9Global Data AssimilationObservations Processing
- Definitions
- Received The number of observations received
operationally per day from providers (NESDIS,
NASA, Japan, Europeans and others) and maintained
by NCEPs Central Operations. Counted
observations are those which could potentially be
assimilated operationally in NCEPs data
assimilation system. Observations from
malfunctioning instruments are excluded. - Selected Number of observations that is selected
to be considered for use by the analysis (data
numbers are reduced because the intelligent data
selection identifies the best observations to
use). Number excludes observations that cannot
be used due to science deficiencies. - Assimilated Number of observations that are
actually used by the analysis (additional
reduction occurs because of quality control
procedures which remove data contaminated by
clouds and those affected by surface emissivity
problems, as well as other quality control
decisions)
10Global Data AssimilationObservations Processing
(cont)
2002 July 2005 Notes November 2005 Operations
Received 123 M 169.0M Nov. 2005 increase attributed to additional AIRS, MODIS winds, NOAA-18 and NOAA-17 SBUV data 236.1 M
Selected 19 M 23.6 M 26.9 M
Assimilated 6 M 6.7 M 8.1 M
11Overview
- Introductory remarks
- Observations and Data Assimilation (DA)
- History of NEXRAD data use in DA, including
precipitation assimilation (Parrish, Lin) - CONUS impact study (Alpert)
- Hurricane impact study (Liu)
- Summary and outlook
12VAD WindsBill Collins, D. Parrish
- VAD winds reinstated 29 March 2000
- First used by RUC (June 1997) and NAM-Eta (July
1997) - Withdrawn from operations (Jan. 1999) due to
problems with observation quality - Error sources
- Migrating birds (similar to errors in wind
profilers) - Southerly wind component too strong (fall)
- Northerly wind component too strong (spring)
- Characteristic altitudes and temperatures
- 5 of all winds
- Winds of small magnitude
- Source unknown
- 8 of all winds
- Outliers (large difference from model guess)
- Source unknown
- 7 of all winds
- Random, normally distributed, errors
- 2x magnitude expected from engineering error
analysis - Acceptably small
- Total 20 of observations have unacceptable
errors
13Stage II and Stage IVMulti-sensor
Precipitation AnalysesYing Lin
Stage II
- Generated at NCEP
- Hourly radar and from hourly gauge
- reports
- First generated at 35 minutes
- after the top of the hour
- 2nd and 3rd at T6h and T18h
- No manual QC.
Stage IV
- National mosaic assembled
- at NCEP
- Input hourly radargauge analyses by
12 CONUS River Forecast Centers (RFCs) - Manual QC by RFCs
- Product available within an hour
- of receiving any new data
- http//www.emc.ncep.noaa.gov/mmb/
- ylin/pcpanl/stage2/
14Assimilation of Precipitation Analyses24 May
2001 Ying Lin
- Motivation
- Direct model precipitation contains large biases
- Impacts all aspects of hydrological cycle
- Soil moisture and surface latent heat flux
particularly impacted - Real-time Stage II precipitation analyses are
available - Assimilation technique
- Precipitation nudging technique
- Comparison of model and observed precipitation
- Change model precipitation, latent heating and
moisture in consistent way dependent on ratio
Pmodel/Pobs - Expected improvements in NAM-Eta
- Short-term (0-36 h) precipitation
- Cycled soil moisture and surface fluxes
- 2 meter temperature
- No negative impact on other predicted fields
1524 May 2001 (cont)
- Impacts as expected
- Significantly improves the model's precipitation
and soil moisture fields during data assimilation
(e.g. North Americal Regional Reanalysis) - Often has a significant positive impact on the
first 6 hours of the model's precipitation
forecast - Occasional positive impact on precipitation
forecasts 24h and beyond - Modestly positive impact on forecast skill scores
- Not used in snow cases due to low observational
bias - No negative impact is seen on the model forecast
temperature, moisture and wind fields
Observed Precipitation 6-h Model Forecast Without
Assim. With Assim.
168 July 2003 NAM-Eta Upgrade
- Stage II and Stage IV hourly analyses merged
precipitation assimilation - Analyses must arrive before data cutoff (H
115) - Quality control added to merged product
- Assimilation of Level 3 NEXRAD 88D radial wind
data - Time and space averaged data (compression)
- First 4 radar tilts (0.5, 1.5, 2.5, and 3.5
degrees) the NIDS feed (14) - Hourly (18)
- Horizontal resolution of
- 5 km radially (120)
- 6 degrees azimuthally (16)
- Overall compression 13840
- Quality control applied from VAD winds, including
migrating bird contamination - These radial wind runs show little positive or
negative impact in the verification statistics,
so it is certainly safe to include these winds
treated this way in the 3DVAR - First implementation do no harm
17Overview
- Introductory remarks
- Observations and Data Assimilation (DA)
- History of NEXRAD data use in DA, including
precipitation assimilation (Lin, Parrish) - CONUS impact study (Alpert)
- Hurricane impact study (Liu)
- Summary and outlook
18CONUS Impact Study with Level 2.5 Winds
- Why compression?
- Observations contain a high degree of redundancy
- Communications cannot (until recently) handle the
data volume for unprocessed observations - NCEP algorithm for winds processing (Superobs)
installed on NEXRAD - Compression parameters can be modified without
impacting code change management - Standard NCEP processing algorithm
19Adaptable Parameters for the Level 2.5 Superob
Product
Parameter Default Range Time
Window 60 minutes 5-90 min Cell Range
Size 5 km 1-10 km Cell Azimuth Size 6
degrees 2-12 deg Maximum Range 100
km 60-230 km Minimum Number of points
required 50 20-200 Same as Level 3
products except for additional tilts and
processing algorithm
20Impact on Precipitation Forecasts
8-20 June 2004 (2 weeks)
24-h accumulated precipitation equitable threat
score (upper) and bias (lower) from Eta 32-km
60-h forecasts from 8JUN2004 20JUN2004 for
various thresholds in inches. The solid line ()
are the radial wind super-ob level 2.5 experiment
and the dash is the Eta control (?) with NIDS
level 3.0 super-obs.
21Impact of Level 2.5 Obs on Forecast Geop. Height
Improved RMS scores for height
Height RMS
Height Bias
Small improvements in upper troposphere No
degradation
22Impact of Level 2.5 Obs on Forecast Winds
No degradation in Vector wind slightly better
near jet levels.
Wind RMS Vector Error
Small improvement in upper troposphere
RMS vector wind errors against RAOBS over the
CONUS from Eta 32-km 60-h forecasts, 8JUN2004
20JUN2004 (24 forecasts). The dash line is the
radial wind super-ob Level 2.5 and the solid line
is the Eta control with NIDS level 3.0
super-obs.
23Impact of Level 2.5 Obs on Forecast Precipitation
24 h Forecast
Control
Obs Radar
Level 2.5
Difference
24Summary Level 2.5 Winds
- Winds received operationally from every radar
site (April 2003) - Improved precip, height and wind scores (none
from Level 3) - Data processing impacts forecast scores
- Subjective evaluation shows positive impact
- Quality control issues remain
- Difficult to solve with processing at radar sites
- Motivates transmission of full data set to NCEP
and robust QC effort at central site
25Overview
- Introductory remarks
- Observations and Data Assimilation (DA)
- History of NEXRAD data use in DA, including
precipitation assimilation (Lin, Parrish) - CONUS impact study (Alpert)
- Hurricane impact study (Liu)
- Summary and outlook
26Airborne Doppler RadarData Analysis in HWRF Model
- Q. Liu, N. Surgi, S. Lord
- W.-S. Wu, D. Parrish
- S. Gopal and J. Waldrop
- (NOAA/NCEP/EMC)
- John Gamache
- (AOML/HRD)
27Background
- Initialization of hurricane vortex
- GFDL model uncycled system
- Spin-up from axisymmetric model with forcing
from observed parameters - Surface pressure
- Maximum wind
- Radii of max. wind, hurricane and T.S. winds
- Increase of observations in hurricane environment
- Dropsondes
- Satellite winds
- Scatterometer (QuikSCAT)
- Sounding radiances (AMSU, AIRS, HIRS)
- Dopper radar (research)
- 13 M program to add Doppler radar to GIV
aircraft - Use of NEXRAD data in landfall situations
- Hurricane is the only system uninitialized from
observations at NCEP
28Cycled Hurricane AnalysisSummary
- Capture short-term intensity changes
- Account for storm motion
- 6 hourly cycling
- Use all available observations
- When no observations, try to correct model
intensity with axisymmetric correction - First time use bogus vortex
293D-VAR Doppler Radar Data Assimilation
- Data Quality Control
- John Gamache (HRD)
- Superobs
- James Purser, David Parrish
- Dx10km, Dy10km, Dz250 m
- Minimum number of data 25
- NCEP Gridpoint Statistical Interpolation (GSI)
analysis - Hurricane Ivan 2004 September 7
- Mature storm
-
30Guess Field
31(No Transcript)
32(No Transcript)
33(No Transcript)
34(No Transcript)
35(No Transcript)
36Future Work
- Run more model forecast using the new analysis
for weak storms - Study the impact of the airborne radar data on
hurricane track and intensity forecasts,
particularly for weak storms - Run HWRF complete cycling system during 2006
hurricane season
37Summary and Outlook
- Use of NEXRAD wind data has proceeded in
incremental steps over the past 9 years - Level 3 ? Level 2.5 ? Level 2
- Use of reflectivity for
- Precipitation analyses
- Model initialization
- Remaining issues
- Quality control
- Model initialization (increasing system
complexity)
38Summary and Outlook (cont)
- June 2005 - Implemented Level 2.5 (superobbed)
data - June 2006 Hierarchical radar data ingest for
WRF-NAM - Level 2.0 (full resolution radial winds)
- Level 2.5 (superobbed winds)
- Level 3 (NIDS feed)
- Precip. Assimilation impacts land surface only
- Prototype data assimilation for hurricane
initialization - Airborne Doppler radar
- Coastal radar
- 2004 cases as prototype
- 2006 cases will be run as demonstration project
- Integrating quality control codes into NCEP North
American Model (NAM) run - Visiting scientist hired (on board at EMC 30
June, 2006) - Winds - expect steadily increasing impact
- Reflectivity - long term project requiring
advanced data assimilation techniques
39ThanksQuestions?
40Doppler Velocity Data Quality Problems
1. Noisy fields (due to small Nyquist
velocity) 2. Irregular variations due to scan
mode switches 3. Unsuccessful dealiasing 4.
Contamination by migrating birds 5. Ground
clutters due to anomalous propagation (AP) 6.
Large velocities caused by moving vehicles
AP 7. Sea Clutter
EMC Working with NSSL and CIMMS to address all QC
issues
41Level 2 Radar Data Assimilation Strategy
- NAM assimilates Level 2 data 20 June
- QC codes are being ported from NSSL CIMMS
- Address all QC issues
- Visiting Scientist on board at NCEP (30 June)
- Former NSSL scientist
- Prior experience with codes
- Tuning and case studies
- Assimilating reflectivity will be a long-term
project, dependent on advanced data assimilation
techniques
42Milestone and Time Table
FY06 Task 1. Complete porting existing
reflectivity QC C code executable together with
the NCEP Fortran code into single compliable
executable. Shunxin Wang (QC C code developer)
will work on this task as early as possible to
meet NCEP's immediate needs. FY06 Task 2.
Complete Phase 1 (by Sept, 2006) Complete
initial stages of Phase 2 (Dec, 2006) Pengfei
Zhang and Shunxin Wang will work together to
design the NCEP/NSSL FORTRAN QC code. Li Wei
working with Shunxin will combine various DA
approaches towards an integrated Fortran DA for
NCEP. Code sets developed during the above two
phases will be ported, tested, and refined on
NCEP computers by Shun Liu (and others at
NCEP). FY07 and beyond TBD
43Flowchart of Real-time Migrating Bird
Identification
Raw data
Calculate QC parameters
Night?
no
yes
Bayes identification and calculate posterior
probability
P(B xi) gt0.5
no
yes
Bird echo
Next QC step
44Current NSSL Radar Data QC packages
Doppler Velocity Vr QC
Reflectivity Z QC
Input Level II data
Input Fortran Data Structure
Pure clear air vol.echo removal
Ground Clutter Detection
Hardware test pattern vol.removal
Dealiasing
Speckle filter
Tilt-by-tilt Vr QC (bird, noisy Vr etc.)
Sun strobe filter
Pixel-by-pixel 3D Z QC (clear air, bird, insect,
AP, sea clutter, interference etc.)
Output Fortran Data Structure
Fortran code
C code
45Phase I NSSL/NCEP Fortran QC package
Reflectivity Doppler Velocity QC
Combined QC Filter from C code
Input Fortran Data Structure
Pure clear air vol. echo removal
Hardware test pattern removal
Ground Clutter Detection
Speckle filter
Dealiasing
Sun strobe filter
Rewrite in Fortran and integrate into Vr QC
Tilt-by-tilt Vr QC
Output QCed Z and Vr Fortran Data Structure
Optimize the entire package
Fortran code
46Phase II NSSL/NCEP Fortran QC package
Reflectivity Doppler Velocity QC
Input Fortran Data Structure
Combined QC Filter
New Dealiasing Algorithm (DA)
Ground Clutter Removal
- Build test-case data base for comparing different
DAs. - b. Develop optimum Fortran DA code set based on
comparisons with research and operational DA
approaches.
Tilt-by-tilt Z Vr QC
Output QCed Z and Vr Fortran Data Structure
Fortran code
47Phase III NSSL/NCEP Fortran QC package
Reflectivity Doppler Velocity QC
Input Fortran Data Structure
Combined QC Filter
Ground Clutter Detection
- Upgrade Vr QC to Z Vr QC.
- Improve tilt-by-tilt QC based on Bayes
statistics. - Expand raw ground truth data base optimize QC
thresholds for radars at different regions (in
terms of geographical and climatologic
conditions).
New Dealiasing
Tilt-by-tilt Z Vr QC
Output QCed Z and Vr Fortran Data Structure
Fortran code
48Strategies for Developing Unified Fortran QC
package
- Prioritize development phases based on
anticipated QC skill and difficulties for each
phase. - Modularize individual components and routines
(with on/off options) to facilitate CPU
performance and optimization on NCEP computers. - Prioritize parameters in the QC package in order
to simplify or enhance the package to fit the
requirement and associated resources. - Develop and maintain QC archive important and/or
challenging cases for comparing and testing.
Includes collecting DA cases to assess different
DA schemes, towards a optimum single DA code set. - Monitor and capture problematic cases, expand
raw ground truth data base, and optimize QC
thresholds for each properly-classified category
(such as VCP, diurnal, seasonal, regional, etc).
49Noisy Vr field (0022UTC)
50Problems in Operational Dealiasing
Level-II raw data
Level-III NIDS
KBUF
KBUF
raw
dealiased
51Review Three-step Dealiasing for Level-II
Velocities
Raw data
3-Step Noise Remove (BA88 )
Step 1
Select circles, Mod-VAD (u0,v0), Pre-dealiasing
VAD (u0,v0), Vertical check
Horizontal averaging variance check
Step 2
Calculate Vr (refined reference)
Quality check (flag0, 1 or 10)
Dealiasing with Vr (skip if flag 0 or 1)
Step 3
Dealiasing with continuity check
Output
Adopted
52Polarimetric (KOUN) vs WSR-88D (KTLX)
KOUN
KTLX
Bird
Storm
rHV
Reflectivity
May 24 2003 0852UTC
53Comparison of rain and bird echoesDoppler
Velocity (zoom in)
KPBZ
KTLX
Rain
Bird
54Jung and Zapotocny JCSDA Funded by NPOESS IPO
Satellite data 10-15 impact