Numbriliste 4D andmete esmane ttlemine

1
Numbriliste 4D andmete esmane töötlemine
4D võrguandmed (gridded data) Formaadid uurija
formaat GRIB NetCDF GRIB ekstraheerimine wgr
ib ? SURFER .GRD Esmane töötlemine
GRADS Ülesanded Kaardid HIROMB andmetest
wgrib GRADS
2
Okeanograafiliste andmete iseloom üldjuhul
juhuslikud väljad ?(x,y,z,t) erijuhud 1D
aegrida ?(x0 ,y0 ,z0 ,t) 1D sügavusprofiil
?(x0 ,y0 ,z,t0 ) 1D horisontaalprofiil ?(x,y0
,z0 ,t0 ) ?(x0 ,y,z0 ,t0 ) 2D horisontaalne
kaart ?(x,y,z0 ,t0 ) 2D aeg-sügavuslõige ?(x0
,y0 ,z,t) 2D ruumiline lõige ?(x,y0 ,z,t0
) ?(x0 ,y,z,t0 )
Esitus funktsiooni graafik
Esitus kaart
3
Võrguandmed (gridded data)
x, y, z, t diskreetsete sammudega Erivariandid
x, y tasapinnaline x,y (sh pööratud) Lat, Lon
(sh pööratud) ortogonaalne kõverjooneline (Lat,
Lon on selle erijuht) Erivariandid
z sügavused/kihid (sageli muutuva kihi
paksusega) sigma-koordinaat (jälgib sügavusi
võrdsete vahedega) isopükniline (jälgib
liikuvaid samatiheduspindasid) hübriidne
(eelnevate kombinatsioon)
vt ka http//www.msi.ttu.ee/elken/SOL_07.ppt htt
p//www.msi.ttu.ee/elken/OceanLim_Notes15.pdf
4
uurija formaat
Programmide näited
Uurija kirjutab ise näiteks FORTRANis talle
sobivas formaadis ja loeb hiljem samal kujul
sisse Kirjutamisel ja lugemisel tuleb üldjuhul
kasutada täpselt samasugust andmete
kirjeldust Puudub info andmete sisu ja aja kohta
(selle saab panna küll faili nimesse) ning
füüsilise/geograafilise ruumi kohta
DIMENSION T(200,200,10) !..arvutused OPEN(20,and
med.dat, FORMBINARY) WRITE(20) T CLOSE(20) END
DIMENSION T(200,200,10) OPEN(20,andmed.dat,
FORMBINARY) READ(20) T CLOSE(20) !..arvutused
END
5
GRIB (GRIdded Binary)
GRIB (GRIdded Binary) is a mathematically concise
data format commonly used in meteorology to store
historical and forecasted weather data. It is
standardized by the World Meteorological
Organizations Commission for Basic Systems,
known under number GRIB FM 92-IX, described in
WMO Manual on Codes No.306. Currently there are
two versions of GRIB, first edition (current
sub-version is 2) is used operationaly world-wide
by all meteorological centers, for Numerical
Weather Prediction output (NWP). A newer
generation was introduced, known as GRIB second
edition, but it is used only by few centers and
in many cases not for operational broadcast. Some
of the second-generation GRIB are used for
derived product distributed in Eumetcast of
Meteosat Second Generation. GRIB superseded the
Aeronautical Data Format (ADF).
wikipedia
6
GRIB Software applications WGRIB Command line
based program to manipulate, inventory and decode
GRIB files Picogrib GRIB edition 1 C-language
(FORTRAN callable) free decoding package
compatible to some extent with ECMWF GRIBEX
routine NCEP codes free software (C and FORTRAN
library) for decoding and encoding data in GRIB
edition 1 format NCEP codes free software (C and
FORTRAN library) for decoding and encoding data
in GRIB edition 2 format (some template only)
Python grib2 module reads GRIB 2 files. JGrib -
Jgrib is a free library for reading GRIB files in
Java. Meteosatlib - Meteosatlib is a free
software C library and set of tools to convert
satellite images between various formats it can
read and write GRIB data, and its GRIB
encoding/decoding library can be used standalone.
wikipedia
7
GRIB Software applications The NCAR Command
Language can be used to read, analyze and
visualize GRIB data, as well convert it to other
gridded data formats. PyNIO is a Python
programming language module that allows read
and/or write access to a variety of data formats
using an interface modelled on netCDF. degrib
(aka NDFD GRIB2 Decoder) is a reader for GRIB 2
files. wgrib2 is a reader for GRIB 2 files.
GRIB API is an API developed at ECMWF to decode
and encode GRIB edition 1 and 2 data. A useful
set of command line tools is also included.
Ugrib Ugrib is a no cost graphical GRIB viewer
designed for reading GRIB1 files. The website
GRIB.US also aims to provide education on the
prudent and safe use of GRIB data for forecasting
weather Xconv/Convsh Xconv is a graphical tool
for displaying and converting gridded data, and
is available for most operating systems. Convsh
is the command-line equivalent.
8
GRIB Format The GRIB product is broken down into
several subsections
Tables Originating Centers Process/Model ID
Grid Indentifiers GDS/BMS Flag Parameter and
Units Level or Layer Forecast Time Units Time
Range Indicator Data Representation Type
Resolution and Component Flags Scanning Mode
Flag BDS Flag
Indicator Section Product Definition Section
(PDS) Grid Description Section (GDS) - optional
Bit Map Section (BMS) - optional Binary Data
Section (BDS) End Section PDS, GDS, BDS
contain tables
Andmeid näitab ja ekstraheerib utiliit
wgrib (töötab peaaegu kõigil platvormidel)
9
GRIB formaadi iseärasusi

• Isekirjeldav, äärmiselt kokku pakitud kiire
  andmeülekande jaoks.
• Väga paindlik.
• Operatiivses okeanograafias ja meteoroloogias
  üldlevinud standard.
• -
• Horisontaalne resolutsioon 0.001 kraadi, mis
  rannikumere jaoks on ebapiisav
• 4D esituse asemel domineerib 2D, kus ühel
  ajahetkel on 3D väljad kirjutatud 2D väljade
  vahelduvaks jadaks.
• Orienteeritud meteoandmetele. Mereprognooside
  puhul tehakse muudatused (üledefineerimised), mis
  lihtsate programmide juures ei toimi. Ka
  keerukama softi puhul esinevad konverteerimisel
  sageli tõrked.
• Saadaval WINDOWS tarkvara (nt OCENS GRIB
  Explorer) mereprognooside faile ei ava.

10
NetCDF (Network Common Data Form)
NetCDF (Network Common Data Form) is a
machine-independent, self-describing, binary data
format standard for exchanging scientific data.
The project homepage is hosted by the Unidata
program at the University Corporation for
Atmospheric Research (UCAR). They are also the
chief source of netCDF software, standards
development, updates etc. The format is an open
standard. The project is actively supported,
currently at version 3, and with plans underway
for a version 4. It is planned to merge the
project in version 4 with the HDF5 data file
format, giving an alternative netCDF interface to
HDF5 files. The format was originally based on
the conceptual model of the NASA CDF but has
since diverged and is not compatible with it.
wikipedia
The freely available distribution contains the
C/C/F77/F90 libraries, (plus tools
ncgen/ncdump) and can be obtained as source code
as a gzipped tar file or compressed tar file, or
see our list of precompiled binaries on the
netCDF 3.6.2 downloads page.
http//www.unidata.ucar.edu/software/netcdf/
11
NetCDF formaadi iseärasusi

• orienteeritud 4D andmetele
• võimalikud suured failid üle 2 GB
• palju lehitsejaid
• otsetugi MATLABi
• suhteliselt lihtne siduda programmeerimiskeeltega
  
• -
• andmete maht suurem kui GRIBil

12
wgrib
Otsi ja lae internetist alla wgrib.exe cygwin1.dl
l
Tee Run cmd ja siis DOS aknas wgrib
http//www.cpc.ncep.noaa.gov/products/wesley/wgrib
.html
13
Laadi wgrib kataloogi loengumaterjalide
hulgast BS01_200709040000000H00M. Tippimise
vähendamiseks kopeeri näiteks failiks BS01.
Vaatame faili sisu
Ülesanne 1 (1)
14
Ülesanne 1 (2)
Ekstraheerime kirje 16 (temperatuur veepinnal, vt
eelmine slaid) tekstifaili
Teisenda 1D tekstifail temp.dat SURFER 2D failiks
temp.grd Laadi wgrib kataloogi loengumaterjalide
hulgast txt2grd.exe ja DOS aknas tee txt2grd
temp.dat
punkte x ja y suunas
tühik (andmete puudumine)
txt2grd.for ! Converts Text output from wgrib to
2D SURFER binary GRD file ! To add geographical
reference, save in SURFER as ASCII file and edit
bounds ! by Jüri Elken ! Usage txt2grd
input.xxx ! Output file is input.grd
Ekstraheeri mõni teine kirje/väli ja joonista
kaart Eesti rannikumeres
15
Geograafiliste koordinaatide määramiseks
tee wgrib bs01 -V
GRD ASCII formaadis asenda
DSAA 752 735 0 751 0 734 6.0172600746155
18.613899230957
16
Grid Analysis and Display System (GrADS)
Center for Ocean-Land-Atmosphere Studies
http//grads.iges.org/grads/
The Grid Analysis and Display System (GrADS) is
an interactive desktop tool that is used for easy
access, manipulation, and visualization of earth
science data. The format of the data may be
either binary, GRIB, NetCDF, or HDF-SDS
(Scientific Data Sets). GrADS has been
implemented worldwide on a variety of commonly
used operating systems and is freely distributed
over the Internet. GrADS uses a 4-Dimensional
data environment longitude, latitude, vertical
level, and time. Data sets are placed within the
4-D space by use of a data descriptor file. GrADS
interprets station data as well as gridded data,
and the grids may be regular, non-linearly
spaced, gaussian, or of variable resolution. Data
from different data sets may be graphically
overlaid, with correct spatial and time
registration. Operations are executed
interactively by entering FORTRAN-like
expressions at the command line. A rich set of
built-in functions are provided, but users may
also add their own functions as external routines
written in any programming language.
17
Data may be displayed using a variety of
graphical techniques line and bar graphs,
scatter plots, smoothed contours, shaded
contours, streamlines, wind vectors, grid boxes,
shaded grid boxes, and station model plots.
Graphics may be output in PostScript or image
formats. GrADS provides geophysically intuitive
defaults, but the user has the option to control
all aspects of graphics output. GrADS has a
programmable interface (scripting language) that
allows for sophisticated analysis and display
applications. Use scripts to display buttons and
dropmenus as well as graphics, and then take
action based on user point-and-clicks. GrADS can
be run in batch mode, and the scripting language
facilitates using GrADS to do long overnight
batch jobs.
18
GRADS kasutamine HIROMB GRIB failide jaoks
Installeerimiseks valida Win32e GrADS is a port
of Brian Doty 's popular Grid Analysis and
Display System (GrADS) to the x86 PC running
32-bit MS Windows operating sytems (win9x, winNT,
win2000), without the need for installing an X
server
Windows keskkonnas tuleb andmed üldjuhul
kopeerida Win32e kataloogi. GRIB failide
lugemiseks peab seal olema (loengumaterjali
hulgast) GRIB andmefail (HIROMB) BS01_20070904000
0000H00M HIROMB indeksfail bs01.idx Andmete
defineerimisfail bs01.ctl Projektsioonide
jaoks skript bsmap.gs
19
GRADS töötab käsuridadega projektsioonideks run
bsmap.gs andmed sisse open bs01.ctl defineerita
kse koordinaadid (üks väärtus või vahemik) kui
kaks vahemikku on tulemuseks kaart, kui üks siis
graafik sügavuste vahemik suuremast väärtusest
väiksemani muutuja nimi võetakse bs01.ctl
failist Temperatuur pinnal set lev 4 d
tmp Soolsus d salty
20
(No Transcript)
21

• Ülesanne 2
• Joonistada kaardid ja/või lõiked/graafikud Eesti
  rannikumeres
• temperatuur TMP
• soolsus SALTY
• veetase DSLM