Access%20HDF5%20Datasets%20via%20OPeNDAP - PowerPoint PPT Presentation

About This Presentation
Title:

Access%20HDF5%20Datasets%20via%20OPeNDAP

Description:

HDF and HDF-EOS Workshop X, Landover, MD. 1 ... HDF and HDF-EOS Workshop X, Landover, MD. 8. Task: 10 elements of 17 MB HDF5 data ... – PowerPoint PPT presentation

Number of Views:93
Avg rating:3.0/5.0
Slides: 23
Provided by: peter1061
Learn more at: http://hdfeos.org
Category:

less

Transcript and Presenter's Notes

Title: Access%20HDF5%20Datasets%20via%20OPeNDAP


1
Access HDF5 Datasets via OPeNDAPs Data Access
Protocol (DAP)
  • MuQun Yang
  • The HDF Group

2
OPeNDAP
  • A software framework that allows simple access
    to remote data
  • Client-server model
  • Servers make the local data accessible
    remotely
  • Clients make applications easily access to
    remote served data
  • Widely used by Earth Science Community
  • URL http//www.opendap.org/

3
The Data Request
  • Data are requested with a URL.
  • http//www.cdc.noaa.gov/cgi-bin/nph-nc/datasets/Re
    ynolds_sst
  • Protocol Machine name OPeNDAP server
    Directory File name

?sst10100900180 Constraint
  • The user can impose a constraint on the data that
    s/he would like to acquire from a data set by
    appending a constraint expression to the end of
    the URL.

4
OPeNDAP Data Types
  • OPeNDAP provides three basic data objects
  • data descriptor structure (.dds) - syntactic
    metadata - rigid
  • data attribute structure (.das) - semantic
    metadata - flexible
  • data (.dods) - the actual data in a binary
    structure

5
OPeNDAP Servers
No fully-supported HDF5 server
6
OPeNDAP Clients
No HDF5 client
7
Why HDF5-OPeNDAP server and client?
Users! Users!! Users!!!
8
A User Case
  • Task
  • 10 elements of 17 MB HDF5 data
  • 200 elements of 10 MB NetCDF data
  • 150 elements of 5 MB HDF4 data

9
General approach
HDF5 data


FTP
HDF4 data


FTP
NetCDF data


FTP
10
Approach with OPeNDAP
HDF5 data


FTP
HDF4 data

server
DAP
client
NetCDF data

DAP
server
11
Conclusion
  • HDF5-DAP server is required
  • HDF5-DAP client is strongly recommended

12
Previous work - A Prototype of DAP2-HDF5 server
  • Incomplete mapping from HDF5 to OPeNDAP
  • No comprehensive test suite
  • Only tested a few examples on two platforms
  • Incomplete documentation

13
An example from SSMI
14
Current HDF5-OPeNDAP Support
HDF5

DAP
Prototyped Server
No Client
15
HDF5-OPeNDAP Project
  • Two-year project awarded by NASA ROSES NRA
  • Funding much less than expected
  • Original proposed work kept
  • with significant contributions from ESDIS
  • Starting date Nov. 15th, 2006

16
Proposed Work software development
  • Implementation
  • HDF5-DAP2 server
  • HDF5-DAP4 server
  • Deployment
  • Release servers to OPeNDAP community

17
HDF5-DAP2 server
  • Update and complete the prototype, including make
    files, instructions, documentation, etc
  • Add a few selected features to handle NASA Aura
    data and NPOESS data
  • Map HDF5 group to DAP(?)
  • Map Object/regional reference to DAP(?)
  • Implement a comprehensive test suite
  • Test on platforms supported by OPeNDAP
  • Demonstrate with sample NASA data

18
Fully-Supported HDF5 Server
HDF5
HDF5 server

DAP
HDF5
HDF5 server
Matlab/Ferret Client

DAP
19
Proposed Work software development
  • Implementation
  • DAP4 to HDF5 conversion utility
  • Deployment
  • Release utility to OPeNDAP community
  • Design study
  • Feasibility of DAP-aware HDF5 library

20
Proposed Work - DAP to HDF5 utility
  • A command line tool that converts DAP to HDF5
    file
  • individual DAP variable
  • subsetting

HDF5
Conversion tool
HDF5 server

DAP
21
Proposed work - documentations
  • DAP4 specification
  • HDF5-DAP4 mapping document
  • DAP4 to HDF5 specification
  • RFC for DAP-aware HDF5 library
  • Submit specifications to ESDSWG

22
Acknowledgement
  • This work was supported basing upon the
    Cooperative Agreement with the National
    Aeronautics and Space Administration (NASA) under
    NASA grant NNG05GC60A.  Any opinions, findings,
    and conclusions or recommendations expressed in
    this material are those of the author(s) and do
    not necessarily reflect the views of NASA. 
Write a Comment
User Comments (0)
About PowerShow.com