RSI at the HDF

1
RSI at the HDF HDF-EOS Workshop VI
October 26, 2004 Aurora, CO
2
RSI Representatives

• Eddie Haskell IDL Software Engineer
• ehaskell_at_RSInc.com
• 8 Years of IDL Experience
• Andy Pursch Technical Sales Engineer Mgr.
  apursch_at_RSInc.com
• 16 Years of IDL Experience
• Worked at NASA Goddard and JPL
• Molly Bantz Product Marketing Specialist
  mbantz_at_RSInc.com

3
Agenda

• RSI Corporate Overview
• Introduction of IDL-HDF5 Write Functionality
• Product Demonstration

4
RSI Overview Locations

• RSI Corporate Headquarters Boulder, Colorado
• RSI Washington D.C. Office
• RSI International Offices
• United Kingdom
• France
• Italy
• RSI Distributors
• Worldwide

5
RSI Overview History

• 1977 RSI founded and IDL launched
• Market for IDL grows extensively and RSI becomes
  a global organization
• 1994 RSI releases ENVI
• 2000 RSI acquired by KODAK
• 2004 RSS group of KODAK and RSI become part of
  ITT Industries
• Space Systems Division

6
RSI Products Services

• IDL Interactive Data Language
• ION, Virtual Machine, Dataminer, Wavelet
  Toolkit, New! DICOM Toolkit
• ENVI Remote Sensing Solutions
• FLAASH, NITF Module, RemoteView Link, ARCGIS
  Reader
• RSIs Global Services Solutions
• Algorithm Development, Consulting, Training,
  Commercialization, SBIR Support

7
RSI Industries Served
RSI products and services provide visualization
and analysis solutions for a wide variety of
industries

• Mining/Oil Gas
• Defense/Homeland Security
• Government
• Academic
• Research Development

• Remote Sensing
• Atmospheric Science /
  Meteorology
• Astrophysics
• Medical Imaging
• Engineering

8
IDL and HDF5
9
IDL 6.1 HDF5

• New for IDL 6.1
• The HDF5 dynamically loadable module has been
  enhanced to include write capabilities.
• Write capabilities include datasets, attributes,
  groups, links, datatypes, and many types of data.

10
IDL 6.1 HDF5

• Prior to IDL 6.1 the HDF5 library included only
  read capabilities
• Also included a graphical HDF5 file browser and a
  parse procedure that would parse an entire HDF5
  file and import it into IDL.

11
IDL 6.1 HDF5

• Added routines from the following HDF5
  interfaces
• Attributes H5A
• Datasets H5D
• File H5F
• Group H5G
• Reference H5R
• Dataspace H5S
• Datatype H5T

12
IDL 6.1 HDF5 H5F

• Updated routine
• H5F_OPEN
• Added WRITE keyword to open the file for reading
  and writing. Default is read only.
• New routine
• H5F_CREATE
• file_id H5F_CREATE(Filename)
• This always overwrites the file if one exists.
  Defaults are used for create and access
  properties.

13
IDL 6.1 HDF5 H5T

• New routine
• H5T_IDL_CREATE
• Datatype_id H5T_IDL_CREATE(Data
• , MEMBER_NAMESvector)
• Creates a datatype based on the FIRST element of
  the IDL data (caution when creating a string
  datatype). Compound datatypes can be created by
  passing in an IDL structure.
• Uses H5Tcopy and H5Tcreate.

14
IDL 6.1 HDF5 H5T

• New routine
• H5T_ARRAY_CREATE
• Datatype_id H5T_ARRAY_CREATE(Datatype_id,
  Dimensions)
• New routine
• H5T_REFERENCE_CREATE
• Datatype_id H5T_REFERENCE_CREATE(/REGION)

15
IDL 6.1 HDF5 H5T

• New routine
• H5T_INSERT
• H5T_INSERT, Datatype_id, Name, Field_id
• This procedure adds a new member to the end of a
  compound datatype.
• New routine
• H5T_PACK
• H5T_PACK, Datatype_id

16
IDL 6.1 HDF5 H5S

• New routine
• H5S_CREATE_SCALAR
• Dataspace_id H5S_CREATE_SCALAR()
• Complements existing routine
• H5S_CREATE_SIMPLE

17
IDL 6.1 HDF5 H5D

• New routine
• H5D_CREATE
• Dataset_id H5D_CREATE(Loc_id, Name,
  Datatype_id, Dataspace_id, CHUNK_DIMENSIONSvecto
  r)
• CHUNK_DIMENSIONS is required for extendable
  datasets
• Default values are used for allocation time, fill
  time, and fill value

18
IDL 6.1 HDF5 H5D

• New routine
• H5D_WRITE
• H5D_WRITE, Dataset_id, Data,
• , MEMORY_SPACE_IDvalue
• , FILE_SPACE_IDvalue
• Data will be converted, if possible, to the
  datatype that was used in the creation of the
  dataset

19
IDL 6.1 HDF5 H5A

• New routine
• H5A_CREATE
• Attr_id H5A_CREATE(Loc_id, Name, Datatype_id,
  Dataspace_id)
• Attributes are limited in size to 16KB

20
IDL 6.1 HDF5 H5A

• New routine
• H5A_WRITE
• H5A_WRITE, Attribute_id, Data
• Data will be converted, if possible, to the
  datatype that was used in the creation of the
  attribute

21
IDL 6.1 HDF5 H5R

• New routine
• H5R_CREATE
• Ref H5R_CREATE(Loc_id, Name
• , DATASPACE_IDvalue)
• Ref is returned as either an integer, if an
  object reference is returned, or a named
  structure, if a dataspace region reference is
  returned.
• Updated routine
• H5R_DEREFERENCE

22
IDL 6.1 HDF5 H5A

• New routine
• H5A_DELETE
• H5A_DELETE, Loc_id, Name
• This function should not be used when attribute
  identifiers are open on Loc_id as it may corrupt
  the file.

23
IDL 6.1 HDF5 H5D

• New routine
• H5D_EXTEND
• H5D_EXTEND, Dataset_id, Size

24
IDL 6.1 HDF5 H5G

• New routine
• H5G_CREATE
• Group_id H5G_CREATE(Loc_id, Name)
• New routine
• H5G_GET_NUM_OBJS
• Value H5G_GET_NUM_OBJS(Loc_id)

25
IDL 6.1 HDF5 H5G

• New routine
• H5G_GET_OBJ_NAME_BY_IDX
• Name H5G_GET_OBJ_NAME_BY_IDX(Loc_id, Index)
• New routine
• H5G_SET_COMMENT
• H5G_SET_COMMENT, Loc_id, Name, Comment

26
IDL 6.1 HDF5 H5G

• New routine
• H5G_LINK
• H5G_LINK, Loc_id, Current_Name, New_Name
• , /SOFTLINK , NEW_LOC_IDvalue
• Combines H5GLink and H5GLink2, calling the
  appropriate one as needed.
• New routine
• H5G_UNLINK
• H5G_UNLINK, Loc_id, Name

27
IDL 6.1 HDF5 H5G

• New routine
• H5G_MOVE
• H5G_MOVE, Loc_id, Src_Name, Dst_Name
• , NEW_LOC_IDvalue
• Combines H5GMove and H5GMove2, calling the
  appropriate one as needed.

28
IDL 6.1 HDF5 H5R

• New routine
• H5R_GET_REGION
• Dataspace_id H5R_GET_REGION(Dataset_id,
  Reference)

29
IDL 6.1 HDF5 H5S

• New routine
• H5S_SET_EXTENT_SIMPLE
• H5S_SET_EXTENT_SIMPLE, Dataspace_id, Dimensions
  ,MAX_DIMENSIONSvector
• New routine
• H5S_SET_EXTENT_NONE
• H5S_SET_EXTENT_NONE, Dataspace_id

30
IDL 6.1 HDF5 H5T

• New routine
• H5T_COMMIT
• H5T_COMMIT, Loc_id, Name, Datatype_id
• A named Datatype can be shared by objects within
  the same HDF5 file, but not by objects in other
  files.

31
IDL 6.1 HDF5

• New routine
• H5_CREATE
• H5_CREATE, Filename, Structure
• The structure is modeled after the structure
  returned from the existing routine H5_PARSE().
• H5_CREATE may not be able to recreate a file when
  given the structure returned from H5_PARSE().
• Limitations Variable length arrays, data types
  not directly supported in IDL, order of items in
  the file, references,

32
IDL 6.1 HDF5

• H5_CREATE
• Structure tags
• _NAME If not supplied the name of the tag will
  be used (Required for top level items.
  Exception if this a top level group a new group
  will only be created if _NAME is supplied)
• _TYPE GROUP, DATASET, ATTRIBUTE,
  DATATYPE, LINK
• _DATA IDL data. Full path to object if LINK.
  Not used for GROUPS
• _COMMENT comment
• _LINKTYPE SOFT or HARD LINK only, default
  is HARD
• SUBSTRUCTURES Items in a group, attributes, etc.

33
IDL 6.1 HDF5

• H5_CREATE
• Dataspaces are all defined as the full extent of
  the data, and datatypes are created automatically
  based on the type of the data.
• Simple Example
• grey_scale byte(bindgen(256)(bytarr(3)1b))
• pal _TYPE'Attribute', _DATAgrey_scale
• dataset _NAME'Hanning', _TYPE'Dataset',
• _DATAhanning(100,200), PALETTEpal
• H5_CREATE, 'myFile2.h5', dataset

34
IDL 6.1 HDF5/EOS
RSI is committed to delivering products to the
scientific data format community. We will
maintain a watch in order to develop the
functionality necessary to support missions and
new formats. We realize that our work is not
done with HDF5-write functionality and so are
beginning the work of scoping and scheduling
development of HDF5-EOS. Respectfully, Harold
Cline Director of Product Management
35
IDL 6.1 HDF5

• Currently in testing. Hopefully will be
  available in mid-November.
• Questions?
• Comments?
• Requests?