Overview of NOAANESDIS GOESR Hyperspectral Sounder Data Compression Study

1
Overview of NOAA/NESDIS GOES-R Hyperspectral
Sounder Data Compression Study

• Bormin Huang, Allen Huang, Alok Ahuja
•  
• Cooperative Institute for Meteorological
  Satellite Studies
• University of Wisconsin-Madison
• 4th MURI Workshop, April 27-28

2

• What is hyperspectral sounding data?
• It is generated from an interferometer (e.g.
  HIS, AERI, CrIS,
• IASI) or a grating sounder (e.g. AIRS).
• It consists of several thousand spectral
  channels that span the
• infrared region on the order of one wavenumber
  or less
• What is hyperspectral sounding data for?
• to retrieve
• - atmospheric temperature, water vapor, and
  trace gases profiles
• - cloud aerosol properties,
• - surface temperature, emissivities, etc.,
• to derive wind from radiance or retrieved water
  vapor fields,
• for better weather and climate prediction.
• Why does it require compression?
• Unprecedented volume of 3D data that consists of
  one spectral
• and two spatial dimensions (10-100 GB per day)
  
• Beneficial to efficient data transfer and
  archive.
• What is new for the data compression society?
• High correlation among remote disjoint channels
  due to the
• absorption of the same absorbing gases.

3

• Lossless Compression Study
• 2D Wavelet-Based Compression Scheme
• JPEG2000 2D IWT ? Bitplane Coding ? Entropy
  Coding
• 3D Wavelet-Based Compression Schemes
• 3D IWT ? 3D EZW ? Entropy Coding
• 3D IWT ? 3D SPIHT ? Entropy Coding
• 3D IWT ? 1D BWT ? Entropy Coding
• 2D Predictor-Based Compression Schemes
• CALIC 2D Gradient-adjusted Prediction ? Entropy
  Coding
• JPEG-LS 2D Nonlinear Prediction ? Entropy
  Coding

4
2D Wavelet Transform
Integer Wavelet Transform (Lifting Scheme)
5
Wavelet based Schemes

• JPEG2000
• A new ISO/IEC (International Organization for
  Standardization/International Electrotechnical
  Commission) compression standard.
• Successor to the DCT (discrete cosine
  transform)-based JPEG algorithm.

IWT with 3 stages (Taubman et. al. 2000)
6
3D Wavelet Tree Coding
3D EZW It uses the spatial hierarchical tree
relationship of the wavelet transform
coefficients for efficient compression. 3D
SPIHT Refinement of the EZW scheme that provides
better compression while having faster encoding
and decoding times.
Parent-child interband relationship and locations
for EZW and SPIHT coding
7
Predictor-Based Schemes

• 2D Context-based Adaptive Lossless Image Codec
  (CALIC)
• Among the nine proposals in the initial
  ISO/JPEG evaluation in July 1995, CALIC was
  ranked first.
• It is considered the benchmark for lossless
  compression of continuous-tone images.

Schematic description of the CALIC encoder
Neighboring pixels used in prediction (Wu et. al.
1997)
8

• 2D JPEG-LS
• Published in 1999 as a lossless compression
  standard of the ISO/IEC.

Neighborhood of JPEG-LS used in prediction
Schematic description of the JPEG-LS encoder
9
Burrows Wheeler Transform

• Block-sorting compression scheme Burrows et al,
  1994
• Rearranges the positions of the data such that
  the few distinct values under the same previous
  context are grouped together in position.

tennessee          tennessee ennesseet         
tennessee nnesseete         
sseetenne nesseeten          etennesse esseete
nn          nnesseete sseetenne         
nesseeten seetennes          esseetenn eetenne
ss          seetennes etennesse         
eetenness tennessee          ennesseet         
    An example of the Burrows-Wheeler
transform. bwt(tennessee) tsennesee. The
matrix on the right is obtained by sorting the
rows of the left matrix in right-to-left
lexicographic order. denotes end of the data
block and can be considered as the smallest
symbol.
10
Ten selected AIRS granules on Sept. 6, 2002
AIRS radiance field at wavenumber 900.3cm-1 for
the selected granules
11
AIRS radiance field at wavenumber 900.3cm-1 for
the selected granules
12
Compression ratios of different algorithms for
the 10 selected AIRS granules
13

• Bias-Adjusted Reordering (BAR) Scheme
• for Data Preprocessing
• Hyperspectral sounder data features strong
  correlations in disjoint spectral regions
  affected by the same type of absorbing gases at
  various altitudes.
• The Bias-Adjusted Reordering (BAR) scheme is
  used for exploring the correlation among remote
  disjoint channels.
• The technique can be used to improve the
  compression ratio of any existing scheme.
• The BAR scheme paper is accepted to be published
  in Optical Engineering.
• We are in the process of patent application.

14
Effect of the BAR scheme on various compression
algorithms for the 10 selected AIRS granules
15
Summary

• In support of the NOAA/NESDIS GOES-R data
  processing studies, we investigated lossless
  compression of 3D hyperspectral sounding data
  using wavelet-based schemes (3D EZW, 3D SPIHT,
  JPEG2000) and predictor-based schemes (CALIC,
  JPEG-LS).
• The performance rank from best to worst in terms
  of compression ratios before the BAR scheme is
  given in the order of JPEG-LS, 3D SPIHT,
  JPEG2000, CALIC, BWT and 3D EZW.
• The performance rank from best to worst in terms
  of compression ratios after the BAR scheme is
  given in the order of JPEG-LS, JPEG2000, CALIC,
  3D SPIHT, BWT and 3D EZW.
• To take advantage of the spectral correlations,
  we applied the BAR scheme to significantly
  improve the compression performance of all the
  compression algorithms.

Acknowledgement This research is supported by
NOAA NESDIS OSD under grant NA07EC0676.