An Interoperability language to connect netCDF and Geographic communities: ncMLGML v' 0'5

1
An Interoperability language to connect netCDF
and Geographic communities ncML-GML v. 0.5
GML And Geo-Spatial Web ServicesConference
2005 July 18th - July 22nd, Vancouver, British
Columbia

• Stefano Nativi, Ben Domenico, John Caron,
  Ethan Davis, and Lorenzo Bigagli
• University of Florence (and IMAA of CNR)
• Unidata Program Center (sponsored by NSF)

2
Outline

• Sample Use Case
• Atmospheric and Ocean Sciences Community Context
• Existing Technologies
• Overall objectives for Bridging Gap
• standard interfaces
• gateway implementation
• Where Does GML Fit?
• What is ncML GML?
• Conclusions
• References
• Paper
• glossary

3
Real-time Local Forecast Use
IDD
Assimilation
Local weather forecast model
Initialization

• Data integration
• atmospheric
• hydrological
• demographic
• infrastructure

Assimilation
Coupled hydro/flood model
4
Hydrological, Demographic, Infrastructure Data in
GIS

• Represent characteristics of discrete objects
• Spatial characteristics change slowly
• Spatial resolution on the order of meters
• Stored in GIS and relational database tables
• Explicit georeferencing
• Use GIS spatial and SQL queries

5
Atmospheric or OceanScience Data

• Represent state of fluids
• Samples of continuous functions of many
  parameters in 3D space and time
• Spatial resolution on the order of kilometers
• Very large binary datasets
• Specialized formats (e.g., GRIB, netCDF, HDF)
• Implicit georeferencing (e.g., CF conventions)

6
5D Datasets
7
Community Context

• Many existing client/server data systems in the
  atmosphere, ocean sciences communities (AIS or
  ESIS)
• Standards evolving from original base in GIS
  community
• Standards based gateway needed between existing
  AIS data systems and traditional GIS world

8
Existing Technology In Oceans/Atmosphere
Communities
Client Application
THREDDS
HTTP
Catalog.xml
NetCDF
IOsp

GRIB
HDF5
NIDS
GINI


9
OGC Data Service with NetCDF Dataset Protocols
Application
WFS
WCS
GML
DTED
GeoTIFF
Static GIS Features
NITF
HDF-EOS

10
Client Application
NetCDF-Java version 2.2
NetcdfDataset
NetcdfFile
HTTP
HDF5
NetCDF-3
IOsp
NetCDF-4
GRIB
NIDS
GINI



11
WCS Client
NcMLGML
getCapabilities
geoTIFF
getCoverage
netCDF
describeCoverage

WCS coverage
NetCDF
GMLgenerator
geoTIFFgenerator
NcML-G metadata
THREDDS catalogs
OPeNDAP
THREDDS catalogs enhanced with NcML-GML
NetCDF/OPeNDAP data server
netCDF objects
THREDDS enhanced catalog generation tools

12
NcML-GML What is it?

• An Abstract and Content Model reconciliation
  schema for ES and GIS info realms
• A Mediation Markup Language between ncML (XML
  representation of netCDF metadata) and GML
• An extension of ncML core schema, based on GML
  grammar

13
The Motivations

• NcML (i.e. netCDF) and GML semantics are not
  completely interoperable
• For example a netCDF dataset can generate lots
  of geo-information Coverages, but only some of
  them are "useful" or "meaningful" for
  visualization using traditional GIS
• To keep ncML and GML loosely coupled
• NcML must be free to change without any direct
  effect on GML, and vice versa.
• To enable ncML and GML interoperability using a
  declarative language (i.e. a mark-up language)
• To support the development of combined" ESGIS
  applications/services, where both communities
  semantics and data models are required
• WCS could be a good example (i.e. a GALEON
  objective)

14
Objectives

• To explicitly mediate from netCDF hyperspatial
  data to GI coverage data models
• To explicitly encode netCDF CF conventions into
  GML-based elements

15
NetCDF dataset content
N-Dimension Coordinate Systems
ltdimensiongt, ltcoordinateSystemgt ltcoordinateAxisgt
ltnetcdf typegt
explicit/semi-implicit/implicit Geometry
ltdimensiongt, ltvariablegt
netCDF multidimensional dataset (e.g. 4/5D
hypercube)
Scalar measured quantities
ltvariablegt
16
Geoinformation coverage content
2D Spatial Coordinate System elev time
explicit/implicit Geometry
Spatial Reference System (SRS)
2Delevtime dataset
Range set
17
NcML-GML Mediation Process
2D elev time Coverages
netCDF hyperspace dataset (3/4/5D)
a Coverage
18
NcML-GML Related GI conceptsin brief

• A netCDF dataset can contain several different
  coverages
• Each coverage is characterized by a domain, a
  range-set and is referenced by a CS/CRS
• Each coverage is optionally described by a
  geographic extent
• Each domain is characterized by a geometry
• Supported domains regular grid domain, irregular
  grid domain and multipoint domain
• Each range-set lists or points set of values
  associated to each domain location
• Supported range-set types scalar range-set and
  parametric range-set

19
Supported data encodings

• Domain
• Regular grid geometry
• Irregular grid geometry
• Multipoint geometry
• Range set
• Scalar range set
• Parametric range set

20
NcML-GML concepts mapping in brief
Semantics level
21
The Content Model

• The content model for ncML-GML is called ncML-G
• ncML-G package diagram

22
The Content Model Netcdf Coverages

• NcML-G UML model for NetCDF coverage data

ltlt
ncML
gtgt
NetcdfType
Metadata directly derived from netCDF dataset.
Metadata structure designed for ES users/systems.
at least one CS object exists
CS name is a unique
CS axis name is a unique
ltlt
neCDF-coverage
gtgt
ltlt
GML-based
gtgt
NetcdfCoverages

NetCDFCoverage
ltlt
GML
gtgt
-
aCoverageFunction

CoverageFunction

0..1

Metadata related to GI aspects, directly derived
from netCDF dataset semantics or extracted from
ncML metadata.
Metadata content is designed for GIS users/systems
(i.e. GML-based).
23
The Content Model Netcdf Coverage

• NcML-G UML model for NetCDF coverage data

24
The Content Model Netcdf Coverage Domain
ltlt
GML-based
gtgt
ltlt
GML-based
gtgt
1..
NetCDFCoverage
CoverageDomain
domain

ltlt
GML
gtgt
-
aCoverageFunction


0..1

characterizedBy
domain

ltlt
GML-based
gtgt
ltlt
GML-based
gtgt
ltlt
GML
gtgt
ltlt
GML
gtgt
NetCDFGridCoverage
NetCDFMultipointCoverage
GridDomain
RectifiedGridDomain
Implicit domain
describedBy
ltlt
GML
gtgt
-
aSRS

SRS
-
ltlt
GML
gtgt
aSRS
SRS
geometry
ltlt
GML-based
gtgt
ltlt
GML-based
gtgt
IrregularGridDomain
RectifiedIrregularGridDomain
characterizedBy
semi-implicit domain
geometry
ltlt
GML
gtgt
domain

MultiPoint
explicit domain geometry
25
The Content Model NetCDF Coverage RangeSet
26
The XML encoding schema

• netcdf-coverage.xsd
• Schema dependencies

ncML
ncML
netcdf
netcdf-cs
ncML Vs GML mediation schema
ncML-GML
netcdf-coverage
GML
GML
feature
grids
temporal
GML
coverage
coordinate ReferenceSystem
27
XML Schema diagram

• netcdfCoverages
• (element)

28
XML Schema diagram

• NetcdfCoverageType
• (abstract complex type)

29
XML Schema diagram

• NetcdfGridCoverageType
• (complex type)

30
XML Schema diagram

• NetcdfMultiPointType
• (complex type)

31
A simple Example CDL

• BADC dataset
• NetCDF CF1 conventions
• CDL version
• netcdf simple
• dimensions
•         latitude 3
•         longitude 2
•         time UNLIMITED // (5 currently)
• variables
•         double time(time)
•                 timestandard_name "time"
•                 timeunits "minutes since
  1994-01-01 000000"
•                 timelong_name "time"
•         float latitude(latitude)
•                 latitudestandard_name
  "latitude"
•                 latitudeunits "degrees_north"
  
•                 latitudepoint_spacing "even"
•                 latitudelong_name "latitude"

32
A simple Example CDL

•         float temp(time, latitude, longitude)
•                 tempstandard_name
  "surface_temperature"
•                 templong_name "Surface
  temperature in degrees C"
•                 tempunits "deg_C"
•                 temp_FillValue 2.e020f
•                 tempvalid_min -80.f
•                 tempvalid_max 60.f
•                 tempcomment "This parameter
  may be erroneous."
• // global attributes
•                 institute "The British
  Atmospheric Data Centre."
•             Conventions "CF-1.0"
•                 source "Model developed in
  conjunction with IPLSPSC."
•                 history "10 Sep 2002 - Created
  by hand.\n",
•                            "18 Mar 2003 -
  Modified by feet.\n"
•                 title "Model output from
  imaginary model (temperONETER)."
•                 comment "Not very useful
  data."
•                 references "A great report
  somewhere!"
• data

33
A simple ExamplencML core elements
34
A simple ExamplencML coordSyst elements
35
A simple Example ncML-GML coverage elements
Domain
36
A simple Example ncML-GML coverage elements
Range set
Coordinate Reference System
37
The Interoperability Experiment

• Java API for ncML-GML ver. 0.5 is under
  development
• NcML-GML 0.5 will be assessed in the framework of
  GALEON IE
• Use WCS for netCDF community

38
NcML-GML 0.5 API

• API Functionality
• Convert from binary netCDF file (netCDF-CF 1.0
  compliant) to ncML-GML document
• Validate ncML-GML document
• With respect to inter-element constraints (not
  enforced by XML Schema Language)
• Convert from ncML-GML document to binary netCDF
  file (netCDF-CF 1.0 compliant)

39
NcML-GML 0.5 API

• Technological aspects
• Declarative programming (e.g. XSLT) falls short
  for complex transformation (context-dependent,
  parametric, etc.)
• The API features a hybrid approach declarative
  procedural (Java based)

40
GALEON IE

• OGC Interoperability experiment Geo-interface
  for Air, Land, Earth, Oceans NetCDF
• Main objectives
• Evaluate netCDF/OPeNDAP as WCS data transport
  vehicle
• Evaluate effectiveness of ncML-GML in WCS data
  encoding
• Investigate protocol adequacy for serving and
  interacting with (5D) datasets involving multiple
  parameters (e.g., temperature, pressure, wind
  speed and direction)
• three spatial dimensions
• two temporal dimensions (i.e. actual time,
  forecast time)
• .. suggest extensions to WCS spec.s

41
GALEON Use Case 3

• Objective To access a netCDF 5D dataset through
  WCS-THREDDS gateway getting a ncML-GML
• Return a WCS getCapabilities response based on
  THREDDS inventory list catalogs
• Return a WCS describeCoverage response based on
  ncML-GML data model
• Serve the dataset as a ncML-GML file
• Experiment a WCS client able to access and
  analyze 5D datasets in ncML-GML form

Gateway WCS Server
WCS Client
42
Conclusions

• NcML-GML addresses the netCDF Vs. GI coverage
  data model harmonization and mediation issue
• It implements ncML and GML interoperability,
  keeping them loosely coupled and in a declarative
  way
• It is an example of Mediation Markup Language
• It is particularly suitable to support combined
  ESGIS applications/services
• NcML-GML 0.5 supports grid (regular and
  irregular) and multi-point domains
• API for ncML-GML 0.5 are under development, and
  will be evaluated in the framework of the OGC IE
  GALEON
• NcML-GML will be tested as a means for generating
  WCS describeCoverage and getCoverage responses

43
References

• netCDFhttp//my.unidata.ucar.edu/content/software
  /netcdf/index.html
• GALEONhttp//my.unidata.ucar.edu/content/projects
  /THREDDS/GALEON/GALEON-Activity-Plan.htm
• Acronym Glossaryhttp//my.unidata.ucar.edu/conten
  t/publications/acronyms/glossary.html