The Story of Wavelets: Theory and Engineering Applications

1
Presents
2
The Story of Wavelets
Robi PolikarDept. of Electrical Computer
EngineeringRowan University
3
The Story of Wavelets

• Technical Overview
• ButWe cannot do that with Fourier Transform.
• Time - frequency representation and the STFT
• Continuous wavelet transform
• Multiresolution analysis and discrete wavelet
  transform (DWT)
• Application Overview
• Conventional Applications Data compression,
  denoising, solution of PDEs, biomedical signal
  analysis.
• Unconventional applications
• YesWe can do that with wavelets too
• Historical Overview
• 1807 1940s The reign of the Fourier Transform
• 1940s 1970s STFT and Subband Coding
• 1980s 1990s The Wavelet Transform and MRA

4
What is a Transformand Why Do we Need One ?

• Transform A mathematical operation that takes a
  function or sequence and maps it into another one
• Transforms are good things because
• The transform of a function may give additional
  /hidden information about the original function,
  which may not be available /obvious otherwise
• The transform of an equation may be easier to
  solve than the original equation (recall your
  fond memories of Laplace transforms in DFQs)
• The transform of a function/sequence may require
  less storage, hence provide data compression /
  reduction
• An operation may be easier to apply on the
  transformed function, rather than the original
  function (recall other fond memories on
  convolution).

5
December, 21, 1807
An arbitrary function, continuous or with
discontinuities, defined in a finite interval by
an arbitrarily capricious graph can always be
expressed as a sum of sinusoids J.B.J.
Fourier
6

• Complex function representation through simple
  building blocks
• Basis functions
• Using only a few blocks ? Compressed
  representation
• Using sinusoids as building blocks ? Fourier
  transform
• Frequency domain representation of the function

7
How Does FT Work Anyway?

• Recall that FT uses complex exponentials
  (sinusoids) as building blocks.
• For each frequency of complex exponential, the
  sinusoid at that frequency is compared to the
  signal.
• If the signal consists of that frequency, the
  correlation is high ? large FT coefficients.
• If the signal does not have any spectral
  component at a frequency, the correlation at that
  frequency is low / zero, ? small / zero FT
  coefficient.

8
FT At Work
9
FT At Work
F
F
F
10
FT At Work
F
11
FT At Work
Complex exponentials (sinusoids) as basis
functions
F
An ultrasonic A-scan using 1.5 MHz transducer,
sampled at 10 MHz
12
Stationary and Non-stationary Signals

• FT identifies all spectral components present in
  the signal, however it does not provide any
  information regarding the temporal (time)
  localization of these components. Why?
• Stationary signals consist of spectral components
  that do not change in time
• all spectral components exist at all times
• no need to know any time information
• FT works well for stationary signals
• However, non-stationary signals consists of time
  varying spectral components
• How do we find out which spectral component
  appears when?
• FT only provides what spectral components exist ,
  not where in time they are located.
• Need some other ways to determine time
  localization of spectral components

13
Stationary and Non-stationary Signals

• Stationary signals spectral characteristics do
  not change with time
• Non-stationary signals have time varying spectra

Concatenation
14
Stationary vs. Non-Stationary
X4(?)
Perfect knowledge of what frequencies exist, but
no information about where these frequencies
are located in time
X5(?)
15

Shortcomings of the FT

• Sinusoids and exponentials
• Stretch into infinity in time, no time
  localization
• Instantaneous in frequency, perfect spectral
  localization
• Global analysis does not allow analysis of
  non-stationary signals
• Need a local analysis scheme for a
  time-frequency representation (TFR) of
  nonstationary signals
• Windowed F.T. or Short Time F.T. (STFT)
  Segmenting the signal into narrow time intervals,
  narrow enough to be considered stationary, and
  then take the Fourier transform of each segment,
  Gabor 1946.
• Followed by other TFRs, which differed from each
  other by the selection of the windowing function

16
Short Time Fourier Transform(STFT)

• Choose a window function of finite length
• Place the window on top of the signal at t0
• Truncate the signal using this window
• Compute the FT of the truncated signal, save.
• Incrementally slide the window to the right
• Go to step 3, until window reaches the end of the
  signal
• For each time location where the window is
  centered, we obtain a different FT
• Hence, each FT provides the spectral information
  of a separate time-slice of the signal, providing
  simultaneous time and frequency information

17
STFT
Time parameter
Frequency parameter
Signal to be analyzed
FT Kernel (basis function)
STFT of signal x(t) Computed for each window
centered at tt
Windowing function
Windowing function centered at tt
18
STFT
t-8 t-2
t4 t8
19
STFT at Work
20
STFT At Work
21
STFT At Work
22
STFT

• STFT provides the time information by computing a
  different FTs for consecutive time intervals, and
  then putting them together
• Time-Frequency Representation (TFR)
• Maps 1-D time domain signals to 2-D
  time-frequency signals
• Consecutive time intervals of the signal are
  obtained by truncating the signal using a sliding
  windowing function
• How to choose the windowing function?
• What shape? Rectangular, Gaussian, Elliptic?
• How wide?
• Wider window require less time steps ? low time
  resolution
• Also, window should be narrow enough to make sure
  that the portion of the signal falling within the
  window is stationary
• Can we choose an arbitrarily narrow window?

23
Selection of STFT Window

• Two extreme cases
• W(t) infinitely long ?
  STFT turns into FT, providing excellent
  frequency information (good frequency
  resolution), but no time information
• W(t) infinitely short
• ? STFT then gives the time signal back,
  with a phase factor. Excellent time information
  (good time resolution), but no frequency
  information
• Wide analysis window? poor time resolution, good
  frequency resolution
• Narrow analysis window?good time resolution, poor
  frequency resolution
• Once the window is chosen, the resolution is set
  for both time and frequency.

24
Heisenberg Principle
Time resolution How well two spikes in time can
be separated from each other in the transform
domain
Frequency resolution How well two spectral
components can be separated from each other in
the transform domain
Both time and frequency resolutions cannot be
arbitrarily high!!! ? ?We cannot precisely know
at what time instance a frequency component is
located. We can only know what interval of
frequencies are present in which time intervals
25
The Wavelet Transform

• Overcomes the preset resolution problem of the
  STFT by using a variable length window
• Analysis windows of different lengths are used
  for different frequencies
• Analysis of high frequencies? Use narrower
  windows for better time resolution
• Analysis of low frequencies ? Use wider windows
  for better frequency resolution
• This works well, if the signal to be analyzed
  mainly consists of slowly varying characteristics
  with occasional short high frequency bursts.
• Heisenberg principle still holds!!!
• The function used to window the signal is called
  the wavelet

26
The Wavelet Transform
A normalization constant
Translation parameter, measure of time
Scale parameter, measure of frequency
Signal to be analyzed
Continuous wavelet transform of the signal x(t)
using the analysis wavelet ?(.)
The mother wavelet. All kernels are obtained by
translating (shifting) and/or scaling the mother
wavelet
Scale 1/frequency
27
WT at Work
28
WT at Work
29
WT at Work
30
WT at Work
31
Matlab Demos on CWT
32
Discrete Wavelet Transform

• CWT computed by computers is really not CWT, it
  is a discretized version of the CWT.
• The resolution of the time-frequency grid can be
  controlled (within Heisenbergs inequality), can
  be controlled by time and scale step sizes.
• Often this results in a very redundant
  representation
• How to discretize the continuous time-frequency
  plane, so that the representation is
  non-redundant?
• Sample the time-frequency plane on a dyadic
  (octave) grid

33
Discrete Wavelet Transform

• Dyadic sampling of the time frequency plane
  results in a very efficient algorithm for
  computing DWT
• Subband coding using multiresolution analysis
• Dyadic sampling and multiresolution is achieved
  through a series of filtering and up/down
  sampling operations

H
xn
yn
34
Discrete Wavelet TransformImplementation
Down-sampling Up-sampling
Half band high pass filter Half band low pass
filter
G
H
2-level DWT decomposition. The decomposition can
be continues as long as there are enough samples
for down-sampling.
35
DWT - Demystified
Length 512 B 0 ?
gn
hn
Length 256 B 0 ?/2 Hz
Length 256 B ?/2 ? Hz
a1
G(jw)
d1 Level 1 DWT Coeff.
gn
hn
Length 128 B 0 ? /4 Hz
w
Length 128 B ?/4 ?/2 Hz
-?
?/2
-?/2
?
a2
d2 Level 2 DWT Coeff.
gn
hn
2
Length 64 B 0 ?/8 Hz
Length 64 B ?/8 ?/4 Hz
a3.
d3 Level 3 DWT Coeff.
Level 3 approximation Coefficients
36
Implementation of DWT on MATLAB
Choose wavelet and number of levels
Load signal
Hit Analyze button
sa5d5d1
Approx. coef. at level 5
Level 1 coeff. Highest freq.
(Wavedemo_signal1)
37
Applications of Wavelets
38
(No Transcript)
39
Applications of Wavelets

• Compression
• De-noising
• Feature Extraction
• Discontinuity Detection
• Distribution Estimation
• Data analysis
• Biological data
• NDE data
• Financial data

40
Compression

• DWT is commonly used for compression, since most
  DWT are very small, can be zeroed-out!

41
Compression
42
Compression
43
ECG- Compression
44
Denoising Implementation in Matlab
First, analyze the signal with appropriate
wavelets
Hit Denoise
(Noisy Doppler)
45
Denoising Using Matlab
Choose thresholding method
Choose noise type
Choose thrsholds
Hit Denoise
46
Denosing Using Matlab
47
Discontinuity Detection
(microdisc.mat)
48
Discontinuity Detectionwith CWT
(microdisc.mat)
49
Application Overview

• Data Compression
• Wavelet Shrinkage Denoising
• Source and Channel Coding
• Biomedical Engineering
• EEG, ECG, EMG, etc analysis
• MRI
• Nondestructive Evaluation
• Ultrasonic data analysis for nuclear power plant
  pipe inspections
• Eddy current analysis for gas pipeline
  inspections
• Numerical Solution of PDEs
• Study of Distant Universes
• Galaxies form hierarchical structures at
  different scales

50
Application Overview

• Wavelet Networks
• Real time learning of unknown functions
• Learning from sparse data
• Turbulence Analysis
• Analysis of turbulent flow of low viscosity
  fluids flowing at high speeds
• Topographic Data Analysis
• Analysis of geo-topographic data for
  reconnaissance / object identification
• Fractals
• Daubechies wavelets Perfect fit for analyzing
  fractals
• Financial Analysis
• Time series analysis for stock market predictions

51
History Repeats Itself

• 1807, J.B. Fourier
• All periodic functions can be expressed as a
  weighted sum of trigonometric function
• Denied publication by Lagrange, Legendre and
  Laplace
• 1822 Fouriers work is finally published
• 1965, Cooley Tukey Fast Fourier Transform

143 years
52
History Repeats Itself Morlets Story

• 1946, Gabor STFT analysis
• high frequency components using a narrow window,
  or
• low frequency components using a wide window, but
  not both
• Late 1970s, Morlets (geophysical engineer)
  problem
• Time - frequency analysis of signals with high
  frequency components for short time spans and low
  frequency components with long time spans
• STFT can do one or the other, but not both?
  Solution Use different windowing functions for
  sections of the signal with different frequency
  content
• Windows to be generated from dilation /
  compression of prototype small, oscillatory
  signals ? wavelets
• Criticism for lack of mathematical rigor !!!
• Early 1980s, Grossman (theoretical physicist)
  Formalize the transform and devise the inverse
  transformation ? First wavelet transform !
• Rediscovery of Alberto Calderons 1964 work on
  harmonic analysis

53
1980s

• 1984, Yeves Meyer
• Similarity between Morlets and Colderons work,
  1984
• Redundancy in Morlets choice of basis functions
• 1985, Orthogonal wavelet basis functions with
  better time and frequency localization
• Rediscovery of J.O. Strombergs 1980 work the
  same basis functions (also a harmonic analyst)
• Yet re-rediscovery of Alfred Haars work on
  orthogonal basis functions, 1909 (!).
• Simplest known orthonormal wavelets

54
Transition to the Discrete Signal Analysis

• Ingrid Daubechies
• Discretization of time and scale parameters of
  the wavelet transform
• Wavelet frames, 1986
• Orthonormal bases of compactly supported wavelets
  (Daubechies wavelets), 1988
• Liberty in the choice of basis functions at the
  expense of redundancy
• Stephane Mallat
• Multiresolution analysis w/ Meyer, 1986
• Ph.D. dissertation, 1988
• Discrete wavelet transform
• Cascade algorithm for computing DWT

55
However

• Decomposition of a discrete into dyadic
  frequencies (MRA) , known to EEs under the name
  of Quadrature Mirror Filters, Croisier, Esteban
  and Galand, 1976 (!)

56
Transition to the Discrete Signal Analysis

• Martin Vetterli Jelena Kovacevic
• Wavelets and filter banks, 1986
• Perfect reconstruction of signals using FIR
  filter banks, 1988
• Subband coding
• Multidimensional filter banks, 1992

57
1990s

• Equivalence of QMF and MRA, Albert Cohen, 1990
• Compactly supported biorthogonal wavelets, Cohen,
  Daubechies, J. Feauveau, 1993
• Wavelet packets, Coifman, Meyer, and
  Wickerhauser, 1996
• Zero Tree Coding, Schapiro 1993 1999
• Search for new wavelets with better time and
  frequency localization properties.
• Super-wavelets
• Matching Pursuit, Mallat, 1993 1999

58
New Noteworthy

• Zero crossing representation
• signal classification
• computer vision
• data compression
• denoising
• Super wavelet
• Linear combination of known basic wavelets
• Zero Tree Coding, Schapiro
• Matching Pursuit , Mallat
• Using a library of basis functions for
  decomposition
• New MPEG standard

59
The Story of Wavelets