Using the HDFEOS Datablade with an Earth Science Information Partner

1
Using the HDF-EOS Databladewith an Earth
Science Information Partner

• Charles Falkenberg
• ECOlogic Corp.

2
Content

• Overview of ORDBMS and Datablades
• HDF-EOS Datablade Architecture
• Earth Science Information Partnerships (ESIPs)
• Architecture of the EOS-WEBSTER ESIP
• Using the HDF-EOS Datablade in WEBSTER
• Benefits of the HDF-EOS Datablade

3
Datablade Overview and the Informix ORDBMS

• User defined data types and functions
• C structures, managed in the server as first
  class types
• C methods accessed through SQL functions
• Runs in the database server
• Built and sold by third parties
• Similar to DB2 extenders and Oracle cartridges.
• Example
• Point, Line, Polygon
• Buffer(), Area(), Length()

4
Overview of the HDF-EOS Datablade

• Expose the HDF-EOS API in SQL
• Collects many API calls into a single function
• Ingest function for HDF-EOS files
• Inquiry functions for point, swath or grid fields
• Subsetting functions for point, swath or grid
  fields
• Structural metadata stored in tables
• Ids for File, Point, Swath, Grid, and Field
• Easy to query database for structural info

5
Overview of the HDF-EOS Datablade

• The HDF-EOS library runs in the database server
• Unmodified version of library
• HDF-EOS files stored as BLOBs or in the file
  system
• Output to external files - ASCII, Binary, HDF-EOS
• I/O optimized for HDF-EOS patterns
• I/O calls trapped and replaced with custom
  routines
• Compliant with Informix requirements
• Improved buffering strategy

6
HDF-EOS Datablade Architecture
SQL access to HDF-EOS
Datablade Output Files
Datablade Routines
Datablade Structural Data Tables
HDF-EOS C Lib (unchanged)
HDF-EOS Files in Informix
HDF-EOS in File System
Datablade File I/O Emulator
Informix ORDBMS
7
Using the Datablade with anEarth Science
Information Partner

• EOS-WEBSTER - ESIP
• Web-based System for Terrestrial Ecosystem
  Research
• Deliver discipline specific data and tools
• Integrate vector and raster data for modeling
• Raster time series data
• Stored in HDF-EOS files
• Indexed with metadata tables in WEBSTER
• Subset by the datablade
• Custom clipped of datablade output

8
EOS-WEBSTER and the Datablade

• Ingestion of new data
• Insert file into the datablade
• Insert rows in metadata tables
• Stage I Spatial search
• Uses geodetic datablade
• Produces list of matching holdings
• Stage II Subsetting and clipping
• Subset calls to the datablade
• Polygonal clipping of datablade output

9
EOS-WEBSTER and the HDF-EOS Datablade
WEBSTER Custom Data Product Generation
Stage I
Stage II
WEBSTER Custom Data Product
HDF-EOS Archive Files
Datablade Output Binary or HDF-EOS
Informix
10
Advantages of the Blade

• Accomplished without the HDF-EOS libraries
• SQL already in use
• HDF-EOS accessible from multiple languages
• Java, C, Perl, command line
• Structural data available through SQL
• Improved performance
• Ingest is quick
• Subsetting performs better that C API

11
Status of DataBlade Development

• All HDF-EOS data types supported
• Grid, Swath, and Point data
• Enhanced user interface
• Covers all datablade functionality
• New output formats
• ASCII, binary, or HDF-EOS files.
• New and improved documentation is available.
• Available on SGI and SUN platforms

12
How to find out more

• Contact me or Jeanne Behnke
• Charles Falkenberg csfalk_at_ecologic.net, (202)
  218-4100
• Jeanne Behnke jeanne.behnke_at_gsfc.nasa.gov
• Visit the HDF-EOS Datablade Home Page
• http//heineken.gsfc.nasa.gov/hdfeos/
• Download the datablade
• Visit the EOS-WEBSTER site
• http//webster.sr.unh.edu