Status of HDFEOS, Toolkit, and Related Applications

1
Status of HDF-EOS, Toolkit,and Related
Applications

• L-3 Communications GSI
• February 28, 2005

2
Development
Recent HDF-EOS5 /Toolkit

• Changed buffer allocation from Static to Dynamic
  (32KB chuncks) for Structural Metadata.
• Added capability for
• - handling character string data fields
• - suppressing HDF5 and/or HDF-EOS5 error
  reports
• - Szip (Shuffling) compression
• - handling attributes associated with the
  "Geolocation Fields" group in the Swath API
• Added routines to handle aliases for grid objects
  
• Added support for Geographic projection
• Added numerous FORTRAN wrappers inquiry
  routines

3
HDF-EOS5/Toolkit Development (cont.)

• Changed MAXREGION for subsetting from 32 to 1024
• Improved performance by saving Structure Metadata
  buffer size for every opened file, avoiding
  redundant string manipulations
• Added fix to swath subsetting for single scan
  line subset
• Added support for HDF5-1.6.3, Szip1.2
• Added Cylindrical Equal Area Projection from
  EASEGrid
• Fixed Toolkit compilation warnings in Linux
  Enterprise concerning deprecated function sys
  errlist
• Modified PGS_EPH_getAttitRecords.c to detect any
  erroneous/corrupt att. and/or eph. data files

4
To be resolved
HDF-EOS5 / Toolkit Issues

• The test for equality in TOOLKIT for
  float/double values will be improved.
• The routine for getting only the alias list in
  HDF-EOS5 will be added once NCSA adds the needed
  capability to HDF5
• We could not duplicate the orbsim problem
  reported by HIRDLS. Were still looking into the
  problem.

5
ECS Support of HDF-EOS

• OSs Solaris, Irix, HP, AIX, DEC, Windows
  NT/98/2000/XP, Linux (including 64-bit Opteron
  and Itanium), Mac OS X
• Compilers FORTRAN 77/90 g77/pgf90 , C, C,
  gcc, g

6
Software Releases

• Last release August 2004
• - TOOLKIT 5.2.12, HDF-EOS 2.12,
• HDF-EOS5.1.8
• Next release March 2005
• - TOOLKIT 5.2.13, HDF-EOS 2.13, HDF-EOS5.1.9
• - HDF-EOS5 with HDF5-1.6.3, Szip1.2

7
Downloadable Applications

• HE5View (HDF5 - based browser)
• EOSView (HDF4 - based browser)
• HDFView, Java-based browser (HDF4, HDF5, HDF-EOS
  2 and 5 access)
• heconvert (converts HDF4 - based
  Grid/Point/Swath structures to HDF-EOS 5
  equivalents)
• Compliments HDF4 -gt HDF5 conversion tool
• Not tested on all products
• HEG (HDF-EOS to GeoTIFF converter, subsetting,
  reprojection, stitching, etc.)

8
HDFView

• A Java-based browse tool from NCSA for HDF4 and
  HDF5 based data.
• HDF-EOS plugin extends HDFView functionality to
  HDF-EOS data.
• The extension is a plug-in module and does not
  change functionality. All copy, save, editing,
  image display, etc. for HDF4 and 5 objects
  remains.
• The look and feel of HDFView is identical when
  displaying HDF and HDF-EOS objects.
• Users can access a complete breakdown of HDF-EOS
  objects. e.g. grid information, projection
  information, dimensions, datafields, and
  attributes are displayed. 

9
HDFView (Cont.)

• Beta Release V2.1 Jan. 05
• - Provided a separate HDF-EOS plugin Users
  install HDFView 2.1 and integrate the HDF-EOS
  plugin into it.
• - Ability to do "Save as" for HDF-EOS5 modify
  individual values in grid or swath data. Delete
  entire grids or swaths. Save the result.
• - Separate help window for HDF-EOS objects
• - SGI, SUN, Linux and Windows are supported

10
HDFView (Cont.)

• Known Problems, Recent Fixes and Future Plans
• Problem with opening of files with read only
  permission (both he2 and he5 files) Resolved
• Problem with "Show Properties As" when hdf-eos is
  chosen Resolved
• Problem with opening a table when double click on
  a field Resolved

11
HDFView (Cont.)

• To be implemented by end of March
• Correct data reading for "Table" or "Image" in
  2-D or 3-D field when the default dimensions are
  switched
• HDF-EOS plug-in Users Guide cannot be seen in the
  Users Guide window
• Problem with showing data type in field attribute
  window
• Add capability for showing local (field)
  attributes in he2 files written with pure hdf4.
• HDF-EOS5 save does not retain profile
  information.
• Port to Mac OSX (if we can find a machine with
  Java)
• To be implemented by end of June
• HDF-EOS2 saving.
• Ability to cut/paste objects, modify/delete
  attributes.
• Ability to create new grids/swaths

12
HDF-EOS Post Processing (HEG)

• A utility that converts EOSDIS data from HDF-EOS
  format to common Geographical Information System
  (GIS)-compatible formats.
• Two versions available
• A downloadable desktop version
• Access through NASA archive online storage (Data
  Pools)
• Is being implemented also in Stand-alone Data
  Pool, OGC (Open Geospatial Consortium) prototype
  for Web Services Chaining, and DOWS (Deploy OGC
  Web Services) on the DAACs
• Details of internal file structures are
  transparent to users.

13
HEG Functionality

• Subsetting (spatial, field, band)
• Performs conversion to a single-band or
  multi-band GeoTIFFs
• HDF-EOS Swath to HDF-EOS Grid
• HDF-EOS Swath or Grid to GeoTIFF
• HDF-EOS Swath or Grid to generic binary (with
  metadata file)
• Stitching (mosaicing) subsetting
• Running from command line (batch jobs) or with
  Java-based GUI
• Reprojection (to UTM, PS, TM, STP, LCC, LAMAZ,
  GEO, SIN)
• Subsampling (Subsample stacks)

14
HEG Functionality

• Metadata preservation/creation.
• Currently supports MODIS, MISR, and ASTER
  products (gt80 products)
• Allows file selection from users local storage
• Operable on Sun, SGI, Win, Linux (64-bit soon),
  MAC
• Integrated into ECS Data Pool
• Reduces the transfer time of HDF-EOS data sets
  (if subset is requested)
• Provides the end-user with the exact file
  required by their application
• User Interface
• Portable, written in Java.
• Not dependent on COTS (e.g.. IDL).

15
Future Work Proposed

• Add HDF5 support for EOS Aura data.
• Integrate AIRS L2 products.
• Complete AMSR-E product integration.
• Integrate the HDFView browser with HEG or add
  other minimal HDF and/or GeoTIFF browsing
  capability in the HEG GUI.

16
Future Work (cont.)

• Add another interpolation method, Elliptical
  Weighted Average for bowtie correction of MODIS
  SWATH products.
• Add more efficiency to the subsetting function.
• XML generation of .met (metadata files)