An HDF5-WRF module

1
An HDF5-WRF module

• MuQun Yang, Robert E. McGrath, Mike Folk
• National Center for Supercomputing
  ApplicationsUniversity of Illinois,
  Urbana-Champaign

2
What is WRF?

• Weather Research and Forecasting Model
  Framework for Weather Model Community
• Primarily uses sequential I/O with NetCDF

3
Goal

• Provide an optional HDF5 IO module for WRF
• Investigate IO performance of WRF

4
Results

• Map WRF data model to HDF5
• Implemented two WRF IO modules
• Sequential IO module with compression
• Parallel IO module
• Parallel IO module is include in WRF distribution
• Performance studies
• Presented to WRF workshop, conferences

5
Findings

1. Parallel HDF5-WRF module can greatly reduce wall
   clock time in some WRF applications.
2. Sequential HDF5-WRF module can greatly reduce the
   WRF file size with szip or shuffling and deflate
   compression algorithms.
3. Parallel HDF5 library needs to be improved for
   chunking storage.

6
Impact

• Parallel HDF5 WRF I/O is available to modelling
  community
• Included in WRF 2.0 source distribution (May
  2004)
• This investigation has led to improvements in
  Parallel HDF5

7
For more information

• HDF5 WRF IO http//hdf.ncsa.uiuc.edu/apps/WRF-ROM
  S
• WRF http//www.wrf-model.org/

8
Acknowledgements

• This work is part of NSF-funded Modeling
  Environment for Atmospheric Discovery Expedition
  (MEAD) (http//www.ncsa.uiuc.edu/AboutUs/FocusArea
  s/MEADExpedition.html)