Algorithm For Image Flow Extraction Using The Frequency Domain

1
Algorithm For Image Flow Extraction-Using The
Frequency Domain
2
early-warning systems tracking
reconstructing egomotion visual
segmentation super resolution
Image flow extraction what it is good for ?
3
Several methods differential based region
matching based phase based
frequency based - spatiotemporal
filtering
4
the power spectrum of translating texture at
velocity v ( u,v) occupies a tilted plane in
the frequency domain ?t u?x v?y
motion-sensitive gabor energy filters sample
these planes efficiently
Motion in the frequency domain
Wt UWx VWy
5
a Gausssian multiplied by sine wave or cosine
wave (BPF) (?t0 ?x0 ?y0) - the center
frequency, maximum filter output maximum filter
output for translating texture with spatial
frequency (?x0 ?y0) and such velocity v ( u,v
) that the temporal frequency is ?t u?x0
v?y0 ?t0
3D Gabor Energy Filters
6
a family of filters 12 filters with same
bandwith, (?x02 ?y02)0.5 const, but
different spatial and temporal frequencies
each velocity corresponds to different tilt of
the plane thus to different distribution of
filters outputs
7
Pattern flow
gaussian pyramid
HPF
Image sequence
gabor filtering
motion energy
The algorithm
convolution
measured energy
theoretical energy
pattern flow
8
level L sample density reduction by 2L
convolution with gaussian of size (2L2 -3)x
(2L2 -3) sample reduction- expands
image spectrum convolution- lower levels
higher frequencies enhancement, higher
levels lower frequencies enhancement high
level-high velocity, low level-low velocity (?t
u?x) 0-1.25 level 0 1.25-2.25
level 1 2.25-5 level 2
Gaussian Pyramid
9
FFT of image
FFT of pyramided image
10
image motion characterized by changes
changes-higher frequencies, enhance changes by
HPF
HPF Filtering
FFT
11
filtering with 12 gabor sine phase filters 12
gabor cosine phase filters
Gabor Filtering
12
the sum of the squared output of sine phase
filter plus the squared output of sine phase
filter measure of Gabor energy that is
invariant to the phase of the signal
Motion Energy
13
gabor filters are localized both in space-time
and frequency domains, thus the motion energy
is also localized convolution with gaussian -
enhancing center values (most reliable)
decreasing far away from center values (least
reliable)
Convolution with Gaussian
14
convolution with gaussian (25x25x7) divide the
image into 25x25 sized segments, each of one
can move in a different velocity the center
value in the convulved motion energy segment is
the energy caused by a moving object in that
segment
Measured Energy
15
Parsevals theorem is used to derive equation
that predicts the energy of gabor filter in
response to moving texture , Ri(u,v)
Theoretical Energy
16
for each moving object 12 measured energies
12 predicted energies least squares estimate
for u and v minimizes the difference between
the predicted and measured energies minimizing
Gauss-Newton method ...
Pattren Flow
17
accurate for textures in motion containing
spatiotemporal frequencies near the center
frequencies
Results
18
moving gaussians
pattern flow
19
Car moving at velocity (2,-0.2) pixel per frame,
the algorithm result (1.93,-0.16)
20
adaptive center frequencies choosing
combining pyramid levels substitute gaussian
convolution with different method - e.g
relaxation labeling different numerical methods
for minimization DSP realization
Future Improvements
The End