NARR User Issues - PowerPoint PPT Presentation

Loading...

PPT – NARR User Issues PowerPoint presentation | free to view - id: 246a25-YTc0N



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

NARR User Issues

Description:

N and E are ve, S and W are -ve. Obtaining Data: NCEP ... Data going into the analyses. Types, counts, quality of observations ... – PowerPoint PPT presentation

Number of Views:57
Avg rating:3.0/5.0
Slides: 32
Provided by: wwwtEmcN
Category:
Tags: narr | goingover | issues | user

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: NARR User Issues


1
NARR User Issues
  • Wesley Ebisuzaki
  • CPC/NCEP
  • Wesley.Ebisuzaki_at_noaa.gov
  • httpwwwt.emc.ncep.noaa.gov/mmb/rreanl/

2
Topics
  • Files contents, availability, formats
  • Tools BUFR, GRIB
  • Obtaining the data NCEP, NCAR, NCDC
  • Scripting downloads
  • What is reality? Is reanalysis truth? An
    insider's viewpoint.

3
GRIB Files
  • For grids (horizontal fields) such as 500 mb Z
  • Lambert conformal grid 32 km eta grid (221)
  • Have grids of lat/lon, display programs
  • Model grid not available except at NCEP
  • No plans to make available, display programs?
  • Analyses are on-line and forecasts ?
  • 99 of people will use the GRIB files

4
BUFR
  • For all input observations except SST, snow, ice,
    observed precipitation (GRIB)
  • Input obs BUFR files have QC marks
  • Analyses/forecasts at station locations (MOLTS)
  • Much smaller than GRIB files, limited locations
  • Distribution (informal)
  • Includes restricted data
  • Few users (experience from global reanalysis)
  • Often other sources of data (in easier formats)

5
Misc Files
  • Restart files huge, full precision, NCEP-only
  • Statistics
  • Fit of obs to NARR, fit of obs to R2
  • Small, can be made available on-line
  • Observation counts
  • Small, should be put on-line
  • PNG (monitoring plots)
  • Small, should be put on-line

6
Want more files? Don't expect too much.
  • NARR has zero funding.

7
Despair?
  • Global Reanalysis has been running this way for
    years!
  • CPC has been running global reanalysis for its
    own operations. CPC and the outside community
    have shared interest in maintaining a stable
    assimilation system. NCEP has an interest in
    others using its products.
  • However, limited resources, need to be selective,
    work with partners

8
Status
  • Real-time is running (4 days late)
  • Started building 1.5-2 year on-line rotating
    archive of latest analyses at NCEP
  • 1979-2003 merged files at NCAR, NCDC
  • NCDC has 5 TB merged files on-line, trying to
    support the on-line community by NOMADS
  • EMC people are working on other projects

9
Tools BUFR
  • Situation improving
  • GrADS can read bufr directly

10
Tools GRIB
  • GrADS display and computations
  • grib2ctl.pl make GrADS control files
  • wgrib inventory, decode GRIB files, remove
    records from a file (save space)
  • lcgrib regional subset (save space)
  • copygb change grid (save space or neurons)
  • grbpoint.pl grib -gt point value printout (slow)
  • Gempak (?), NCL (?)

11
Example GrADS
  • Make control file (analyses)
  • grib2ctl.pl -rr cpc_rr_mon.197901 gtmy.ctl
  • grib2ctl.pl -rr cpc_rr_mon.y4m2 gtmy.ctl
  • Make index file (analyses)
  • gribmap -0 -i my.ctl
  • Run GrADS
  • grads
  • open my.ctl

12
Example wgrib
  • Use inventories, make files smaller, decode data
    for fortran and C programs
  • See contents of a grib file
  • wgrib (options) cpc_mon_197901
  • To extract records, decode records, see wgrib
    home page.

13
Example lcgrib
  • Regional subset of Lambert conformal grid
  • MAKE SMALLER FILES
  • lcgrib in-grib out-grib west south east north
  • West, south, east and north are in degrees
  • N and E are ve, S and W are -ve

14
Obtaining Data NCEP
  • Rotating archive latest 1.5 2 years of
    analyses
  • http//nomad7.ncep.noaa.gov/ncep_data
  • Other files TBD
  • NCEP not set up to deal with the public requests
  • Go through the web
  • Go through NCDC, NCAR

15
Obtaining Data NCAR
  • Have 1979-2003 merged data
  • Sure they will get more of the data
  • Plans, availability?

16
Obtaining Data NCDC
  • NCDC has the merged data 1979-2003 and want 2004
    ASAP
  • NCDC will use NOMADS to distribute the data to
    the on-line community.
  • NOMADS is project of many institutions including
    NCDC and NCEP for distributing data over the
    internet.
  • NOMADS-NCDC is taking software from NCEP and COLA
    to distribute GRIB data.

17
Data Access at NCDC
  • Custom plots
  • ftp4u access by selected time, field, region
  • http access to raw data (like FTP)
  • GDS (GrADS Data Server) OpeNDAP server (formerly
    DODS), on-line access to data

18
Difficulties in on-line distribution
  • Downloading a 5 TB data set is difficult
  • Need ways to minimize data transfer
  • Plots (maps) work but limited utility
  • GDS needs to be optimized for large datasets
  • ftp4u is a solution

19
FTP4U
  • Only download data that you need
  • Select levels
  • Select variables
  • Select geographic regions
  • Select times

20
Making FTP4U Work
  • Don't request more data than needed
  • Make one big request, not many small requests of
    the same file i.e., make one request to get the
    winds, temp and heights rather than 3 requests.
    Do not request long time series.
  • Shared resource (1 job at a time)
  • Be prepared to script/automate the downloads
  • May be impossible to download in a few hours, may
    require days, weeks and multiple jobs/requests

21
Scripting FTP4U Downloads
  • FTP4U is normally accessed by a browser
  • Anything a browser can do, a script can also do
  • Experience at NCEP is that most plot/ftp4u access
    is scripted (automated)
  • NCDC's modifications to the NCEP provided
    programs make it more difficult to script the
    download but nothing that a cURL user couldn't
    handle. NCDC wants to change!

22
Scripting Downloads cont'd
  • Use cURL, a unix/windows program, to emulate a
    browser
  • Each browser page becomes a call to cURL.
  • With a little RTFM was able get script working
  • sample program will be put on NARR home page

23
Scripting Downloads cont'd
  • Downside steep learning curve, may have no local
    experts for help
  • Upside can make large data transfers possible,
    can be used in other projects (ex. getting fcst
    data automatically, skewing on-line opinion
    polls)
  • Given that we get more data off the web, it's a
    skill that someone in a large group should
    eventually have.

24
Obtaining Data NCDC
  • It's a learning situation for NCDC
  • Give feedback (complain), it's the only way to
    improve the situation. Lessons learned here will
    help the next time you try getting a different
    dataset from NCDC's nomad server.
  • Current issues full disk (new RAID is will be
    installed soon), obtaining time series, scripting
    support

25
What is Reality?
  • Some people consider reanalysis to be truth
  • NARR is new, bound to be warts, already found
    some.

26
What is Reality?
  • Accuracy of reanalysis depends on many factors
  • Data going into the analyses
  • Types, counts, quality of observations
  • Accuracy of previous analysis
  • Accuracy of atmospheric/land model
  • Spatial and temporal filtering
  • NO EASY ANSWER

27
What is Reality? cont'd
  • Accuracy
  • check to the appropriate data counts and biases
  • Need to add our counts to web
  • Need to add fit to data to web
  • check other reanalyses
  • Fit to observations
  • Consider sensitivity to modeling assumptions
  • Sensitivity experiments (data, resolution,
    model), only available to limited of people

28
What is Reality? cont'd
  • Trust but verify
  • Ronald Reagan

29
URLs
  • copygb ftp//ftpprd.ncep.noaa.gov/pub/cpc/wd51we/
    copygb
  • change grid type of grib files
  • copygb ftp//ftpprd.ncep.noaa.gov/pub/cpc/wd51we/
    copygb
  • change grid type of grib files
  • cURL http//curl.haxx.se
  • download URLs, handles post data
  • GEMPAK http//my.unidata.ucar.edu/content/softwar
    e/gempak/index.html
  • graphics, can handle NARR files with some effort
  • GrADS http//grads.iges.org/grads/
  • graphics, computations

30
URLs
  • grbpoint.pl ftp//ftpprd.ncep.noaa.gov/pub/cpc/wd
    51we/wgrib.scripts
  • simple program to extract data from lat/lon
  • grib2ctl.pl http//www.cpc.ncep.noaa.gov/products
    /wesley/grib2ctl.html
  • makes control files for GrADS
  • g95 http//www.g95.org/
  • f95 compiler for copygb
  • HIPHOP http//iacweb.etzh.ch/stall/dominik/hihop/
  • grib support for idl
  • lcgrib http//www.cpc.ncep.noaa.gov/products/wesl
    ey/gribw.html
  • regional subset of Lambert conformal grib files

31
URLs
  • NARR http//wwwt.emc.ncep.noaa.gov/mmb/rreanl/ind
    ex.html
  • home page for NARR
  • NARR rotating archive http//nomad7.ncep.noaa.gov
    /ncep_data
  • NCAR Command Language (NCL) http//ngwww.ucar.edu
    /
  • graphics, computations package that supports grib
  • NCDC data http//nomads.ncdc.noaa.gov/narr_datas
    ets
  • on-line supplier of NARR data
  • wgrib http//www.cpc.ncep.noaa.gov/products/wesle
    y/wgrib.html
  • inventory, slice dice grib files
About PowerShow.com