Two components

1
Introduction

• Two components
• Data Exchange Infrastructure (DEI)
• Spatial Data Model (SDM)
•  
• Together, they make up NCDDC Middleware

2
Introduction to NCDDC Middleware - what it does
and why

• Richard Owens

3
What Is DEI (High-level overview)
  DEI is a client/server framework by which
applications and data repositories can interact.
The goal of the framework is to solve the
problems of Data Access and Data
Representation.   DEI is a CORBA-based set of
interfaces which are used to locate and transfer
geospatial data. It is a client/server
architecture which uses CORBA's IIOP (uses TCP)
as its transport protocol.
4
DEI Concepts

• Gateway
• Server component that adapts DEI interfaces to a
  specific
• data source
• Data source dependent
• DEI Client
• Any software that uses DEI CORBA interfaces to
• communicate with gateways

CORBA
Client
Gateway
5
DEI Concepts (continued)

• Data Flavor (formats)
• Like a MIME type

6

What DEI Is Not
Complete

• Lacks generic query processing
• Poor support for very large datasets

Data Archive/Storage Data Format

• Supports them, though

Web services
(graphic here)
7
History Originally Navy NRL funded project Now
being developed by NCDDC Current Status Java
based gateways via Java GDK Current focus C
gateways Original focus, now lagging behind
Java Future Mature Java GDK, then C GDK Add
generic query processing Add support for very
large datasets Open source

8

• What is the SDM
• Object-oriented library that is used to
  represent and manipulate spatial data objects.
• 3D
• Some other dimensional capabilities, too
• Structured Grids
• Regular, Rectilinear, Curvilinear, Projected
• Unstructured Grids
• Coordinate Systems
• Cartographic, Cartesian, Cylindrical, Spherical
  
• GIS-like structures
• Points, Lines, Polygons, etc.

9

• What is the SDM (continued)
• Java
• 160 classes
• 1050 Unit tests

10
Key Features

• Called the Field Spatial Data Model by its
  developer because it uses a field concept,
  which relates spatial objects to data.
• Fundamental storage is multi-dimensional array.
• Decoupling of data storage from its geometry and
  topology.
• Requires basic metadata (quantity, units,
  validity, accuracy, etc.) for all data.
• Aids in interoperability

11
Key Features (continued)

• Small, consistent API for data access.
• The SDM hides all implementation details of data
  representation behind its abstract interfaces.
• Decouples many issues regarding storage and
  internal representation of the data, while
  providing a consistent interface to it.
• Data may be logically represented one way and
  physically stored another way.
• Many manipulations may be done without actually
  having to have the data physically in memory.

12
Key Features (continued)

• Externalized format for representing datasets
• XML based (FDML)
• Can be generated without the use of the SDM.

13
Field Concept
14
What SDM Is Not
Transport mechanism Storage/Archival Magic.
Wont solve all problems. Just a data
format Complete




15
History EPA project Ported from Eiffel to Java
in 2002



16

• Future
• Documentation
• FY03
• Text data (done)
• Generalize dimensional support beyond existing
  3-D
• (to 2-D, 1-D or N-D)
• Investigate Time dimension support (complete in
  FY04)
• Application Layer Services
• Common Query mechanism
• Subsetting (parameters, spatial)
• Format support (NetCDF, Shapefile)
• FY04
• Aggregation of data types
• Time dimensions support - including subsetting
  in
• Application Layer
• Open Source

17
NCDDC Middleware Objective Develop a middleware
architecture that can be used to tie data and
applications together across a local or wide-area
network in a way that is independent of the data
and application platforms, operating systems, and
formats.
 
18

• NCDDC Middleware Defined
• NCDDC Middleware is the fusion of DEI with SDM
• DEI
• Data Transport
• SDM
• Data Representation
• DEI SDM (NCDDC Middleware)
• Spatial data representation distributed over a
  network
• Common form for data, and the means for
  transporting it

19

DEI/SDM, HTTP/HTML Analogy Transport HTTP,
DEI Content HTML, SDM





20

• Benefits
• Gateways can be created that can transport
  descriptions of the data with embedded links to
  the actual data the actual transport of the
  data is done piecewise, on demand.
• Can have datasets that are larger than memory.
• Caching mechanism improves performance.
• Data is transported in binary form
• Suns XDR format
• Foundation for netCDF
• Fast and efficient
• Since DEI is integrated within the SDM, clients
  can use SDM interfaces to access gateways without
  the need to code to DEIs CORBA interfaces.