Orthogonal Transforms

1
Orthogonal Transforms

• Fourier Walsh Hadamard

2
Review

• Introduce the concepts of base functions
• For Reed-Muller, FPRM
• For Walsh
• Linearly independent matrix
• Non-Singular matrix
• Examples
• Butterflies, Kronecker Products, Matrices
• Using matrices to calculate the vector of
  spectral coefficients from the data vector

3
Orthogonal Functions
4
(No Transcript)
5
Illustrate it for Walsh and RM
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
Mean Square Error
10
(No Transcript)
11
(No Transcript)
12
Important result
13

• We want to minimize this kinds of errors.
• Other error measures are also used.

14
Unitary Transforms

• Unitary Transformation for 1-Dim. Sequence
• Series representation of
• Basis vectors
• Energy conservation

Here is the proof
15

• Unitary Transformation for 2-Dim. Sequence
• Definition
• Basis images
• Orthonormality and completeness properties
• Orthonormality
• Completeness

16

• Unitary Transformation for 2-Dim. Sequence
• Separable Unitary Transforms
• separable transform reduces the number of
  multiplications and additions from to
• Energy conservation

17
Properties of Unitary Transform
transform
Covariance matrix
18
Example of arbitrary basis functions being
rectangular waves
19
This determining first function determines next
functions
20
1
0
21
(No Transcript)
22
Small error with just 3 coefficients
23
This slide shows four base functions multiplied
by their respective coefficients
24
This slide shows that using only four base
functions the approximation is quite good
End of example
25
Orthogonal and separable Image Transforms
26
Extending general transforms to 2-dimensions
27
Forward transform
inverse transform
separable
28
Fourier Transform
separable
29
Extension of Fourier Transform to two dimensions
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
Discrete Fourier Transform (DFT)
New notation
34
(No Transcript)
35
Fast Algorithms for Fourier Transform
Task for students Draw the butterfly for these
matrices, similarly as we have done it for Walsh
and Reed-Muller Transforms
2
Pay attention to regularity of kernels and order
of columns corresponding to factorized matrices
36
Fast Factorization Algorithms are general and
there is many of them
37

• 1-dim. DFT (cont.)
• Calculation of DFT Fast Fourier Transform
  Algorithm (FFT)
• Decimation-in-time algorithm

Derivation of decimation in time
38

• 1-dim. DFT (cont.)
• FFT (cont.)
• Decimation-in-time algorithm (cont.)

Butterfly for Derivation of decimation in time
Please note recursion
39

• 1-dim. DFT (cont.)
• FFT (cont.)
• Decimation-in-frequency algorithm (cont.)

Derivation of Decimation-in-frequency algorithm
40
Decimation in frequency butterfly shows recursion

• 1-dim. DFT (cont.)
• FFT (cont.)
• Decimation-in-frequency algorithm (cont.)

41
Conjugate Symmetry of DFT

• For a real sequence, the DFT is conjugate symmetry

42

• Use of Fourier Transforms for fast convolution

43
Calculations for circular matrix
44
By multiplying
45
W ? Cw
In matrix form next slide
46
w ? Cw
47
Here is the formula for linear convolution, we
already discussed for 1D and 2D data, images
48
Linear convolution can be presented in matrix
form as follows
49
As we see, circular convolution can be also
represented in matrix form
50
Important result
51
Inverse DFT of convolution
52

• Thus we derived a fast algorithm for linear
  convolution which we illustrated earlier and
  discussed its importance.
• This result is very fundamental since it allows
  to use DFT with inverse DFT to do all kinds of
  image processing based on convolution, such as
  edge detection, thinning, filtering, etc.

53
2-D DFT
54
Circular convolution works for 2D images
55
Circular convolution works for 2D images So we
can do all kinds of edge-detection, filtering etc
very efficiently

• 2-Dim. DFT (cont.)
• example

56

• 2-Dim. DFT (cont.)
• Properties of 2D DFT
• Separability

57

• 2-Dim. DFT (cont.)
• Properties of 2D DFT (cont.)
• Rotation

58

• 2-Dim. DFT (cont.)
• Properties of 2D DFT
• Circular convolution and DFT
• Correlation

59

• 2-Dim. DFT (cont.)
• Calculation of 2-dim. DFT
• Direct calculation
• Complex multiplications additions
• Using separability
• Complex multiplications additions
• Using 1-dim FFT
• Complex multiplications additions ???

Three ways of calculating 2-D DFT
60
Karhunen-Leove Transform(KLT)
Covariance matrix
61
(No Transcript)
62

• What happens if ??1?

63
This happens if ??1
64
Category of transforms
These are what I called earlier transforms with
standard butterflies
65
Discrete Cosine Transform (DCT)
This is DCT
66
DCT is an orthogonal transformm so its inverse
kernel is the same as forward kernel
This is inverse DCT
67
(No Transcript)
68
DCT can be obtained from DFT
69

• Discrete Cosine Transform is asymptotically
  equivalent to Karhunen-Loeve

We take a 2N-point DFT
This is why guys in industry believe that only
DCT is worth their work.
70
Properties of DCT real, orthogonal,
energy-compacting, eigenvector-based
71
The eigenvectors of R and Qc are very close
72

• Basis Functions for 1-dim. DCT(N 16)

73
There are many DCT fast algorithms and hardware
designs.
74
There are many DCT fast algorithms and hardware
designs.
Many fast algorithms are available fast algorithm
Lee(1-D), Lee-Cho(2-D) VLSI algorithm
regularity, local interconnection,
moduality ref 1.Nam Ik Cho and Sang Uk Lee,
Fast algorithm and implementation of 2-D
DCT. IEEE Trans Circuits and Systems, vol.
38, no.3, pp.297-305, March 1991. 2.Nam Ik Cho
and Sang Uk Lee, DCT algorithm for VLSI parallel
implementation IEEE Trans. ASSP, vol.
ASSP-38, no. 1, pp.121-127, Jan, 1990.
75

• Discrete Sine Transform(DST)

Similar to DCT.
76

• 1-dim. DST (cont.)
• Basis Functions for 1-dim. DST (N16)

77
Walsh Transform
78
Here we calculate the matrix of Walsh coefficients
79
Here we calculate the matrix of Walsh coefficients
80
Here we calculate the matrix of Walsh coefficients
81
We have done it earlier in different ways
Here we calculate the matrix of Walsh coefficients
82
Symmetry of Walsh
Think about other transforms that you know, are
they symmetric?
83
Two-Dimensional Walsh Transform
84
Two-dimensional Walsh
Inverse Two-dimensional Walsh
85
Properties of Walsh Transforms
86
Here is the separable 2-Dim Inverse Walsh
87
Example for N4
88
odd
even
89
Discuss the importance of this figure
90
Hadamard Transform
We will go quickly through this material since it
is very similar to Walsh
91
(No Transcript)
92
separable
93
Example of calculating Hadamard coefficients
analogous to what was before
94
(No Transcript)
95
(No Transcript)
96
(No Transcript)
97
(No Transcript)
98
(No Transcript)
99
(No Transcript)
100
(No Transcript)
101
Standard Trivial Functions for Hadamard
One change
two changes
102
(No Transcript)
103
Discrete Walsh-Hadamard transform
Now we meet our old friend in a new light again!
104
(No Transcript)
105
(No Transcript)
106
(No Transcript)
107
(No Transcript)
108
Relationship between Walsh-ordered and
Hadamard-ordered
109
Nonsinusoidal orthogonal function
Haar Transform

• Haar transform
• Haar function (1910, Haar) periodic,
  orthonormal, complete

110
(No Transcript)
111
(No Transcript)
112
Slant transform
113
(No Transcript)
114
(No Transcript)
115
(No Transcript)
116
SVD(Singular Value Decomposition)
117
(No Transcript)
118
(No Transcript)
119
2-D linear processing technique
120
(No Transcript)
121
Questions to Students

1. You do not have to remember derivations but you
   have to understand the main concepts.
2. Much software for all discussed transforms and
   their uses is available on internet and also in
   Matlab, OpenCV, and similar packages.

1. How to create an algorithm for edge detection
   based on FFT?
2. How to create a thinning algorithm based on DCT?
3. How to use DST for convolution show example.
4. Low pass filter based on Hadamard.
5. Texture recognition based on Walsh