# EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms - PowerPoint PPT Presentation

PPT – EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms PowerPoint presentation | free to download - id: 247b6c-M2JiO The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms

Description:

### The objective is to estimate the motion vectors from two time sequential frames of the image. ... complexity is still the main drawback of the DCT-based motion ... – PowerPoint PPT presentation

Number of Views:110
Avg rating:3.0/5.0
Slides: 40
Provided by: Qual69
Category:
Tags:
Transcript and Presenter's Notes

Title: EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms

1
EE5359 Spring 2008 Comparative study of Motion
Estimation (ME) Algorithms
• Khyati Mistry
• 1000552796
• 04/29/2008

2
Comparative study of Motion Estimation (ME)
Algorithms
• Objective
• The goal of this project is to do a comparative
study of the different motion estimation
techniques and search algorithms that have been
proposed for interframe coding in moving video
sequences taking into consideration the different
kinds of motion of the moving objects in
successive frames translational, rotational,
zooming etc

3
Comparative study of Motion Estimation (ME)
Algorithms
• Introduction
• The motion estimation module will create a
model for the current frame by modifying the
reference frames such that it is a very close
match to the current frame. This estimated
current frame is then motion compensated and the
compensated residual image is then encoded and
transmitted.

Fig 1 Motion estimation block diagram 1
4
Comparative study of Motion Estimation (ME)
Algorithms
• The objective is to estimate the motion vectors
from two time sequential frames of the image.
• The objective is to estimate the motion vectors
from two time sequential frames of the image. The
motion of an object in the 3-d object space is
translated into two successive frames in the
image space at time instants t1 and t2 as shown
in Fig 2.
• Where
• t1, t2 represent the time axis such that t2 gt
t1
• (X, Y) Image-space coordinates of P in the
scene at time t1
• (X, Y) Image space coordinates of P at time
t2 gt t1
• (x, y, z) Object-space coordinates at a point P
in the scene at time t1

Fig 2Basic geometry for motion estimation 2
5
Comparative study of Motion Estimation (ME)
Algorithms
• Broad Classification of Motion Estimation
Algorithms

Fig 3. Motion estimation algorithms 10
6
Comparative study of Motion Estimation (ME)
Algorithms
• Frequency domain algorithms In these algorithms,
the algorithm is applied on the transformed
coefficients 10. The different
algorithms/techniques used are phase correlation
21, matching in wavelet and DCT domain 17.
• Time domain algorithms These comprise of
1-16.
• Block matching algorithms match all or some
pixels in the current block with the block in the
search area of reference frame based on some cost
function 1-15.
• Feature based algorithms match the meta
information/data of the current block with that
of the block in reference frame 16.

7
Comparative study of Motion Estimation (ME)
Algorithms
Time domain Vs frequency domain algorithms
Fig 4 (a) Conventional video encoder with
motion estimation and compensation in the spatial
domain. (b) Simplified video encoder with motion
estimation and compensation in the transform
domain. 17
8
Comparative study of Motion Estimation (ME)
Algorithms
• In the frequency domain, the phase correlation
algorithm provides accurate predictions but is
based on the fast Fourier transform (FFT), which
is incompatible with current DCT-based video
coding standards and which has high computational
complexity since a large search window is
necessary. Also, it gives poor performance when
block size is small and motion is not pure
translational.
• DCT-based motion estimation simplifies the
conventional DCT-based video coders achieving
spatial redundancy reduction through DCT and
temporal redundancy reduction through motion
estimation and compensation.
• In the conventional DCT-based video coder, the
feedback loop has the following functional
blocks DCT, inverse DCT (IDCT), quantizer,
inverse quantizer, and spatial domain motion
estimation and compensation.
• If motion estimation and compensation are
performed entirely in the DCT domain, IDCT can be
removed from the feedback loop. Therefore, the
feedback loop contains the reduced functional
blocks quantizer, inverse quantizer, and
transform domain motion estimation and
compensation (Fig 4)
• High computational complexity is still the main
drawback of the DCT-based motion estimation and
compensation approach.

9
Comparative study of Motion Estimation (ME)
Algorithms
• Major Aspects of motion estimation techniques
• Reduced computational complexity
• Good visual quality in terms of representation
of true motion
• High compression ratio

10
Comparative study of Motion Estimation (ME)
Algorithms
• The computational complexity of a motion
estimation technique can be determined by three
factors
• The search algorithm decides the overall
computational complexity and accuracy of motion
estimation
• Search area the span of search window to find
the best match
• Cost function the different cost metrics used to
find the best match

11
Comparative study of Motion Estimation (ME)
Algorithms
• Cost Functions
• The different cost functions that are used to
find the best possible match are listed 10
• CCF (cross correlation function)
• CCF(dx,dy)
• (MVx, MVy) max (dx,dy)?R2 CCF(dx,dy)

12
Comparative study of Motion Estimation (ME)
Algorithms
• MSE (mean square error)
• MSE(dx,dy)
• (MVx, MVy) min (dx,dy)?R2 MSE(dx,dy)

13
Comparative study of Motion Estimation (ME)
Algorithms
• MAE (mean absolute error)
• MAE(dx,dy)
• (MVx, MVy) min (dx,dy)?R2 MAE(dx,dy)

14
Comparative study of Motion Estimation (ME)
Algorithms
• SAD (sum of absolute differences)
• (MVx, MVy) min (dx,dy)?R2 SAD(dx,dy)

15
Comparative study of Motion Estimation (ME)
Algorithms
• Motion estimation Algorithms
• 1. Full search
• This is the brute force technique in which each
block is matched with the current block to find
the best possible match.
• It is a very simple and accurate technique.
• Not efficient because it involves high
computational cost.
• So more efficient algorithms were proposed but
there was always a tradeoff between efficiency
and image quality.
• Fig 5 Full search (Brute force
technique)

16
Comparative study of Motion Estimation (ME)
Algorithms
• 2. Fast motion estimation algorithms
• To meet the requirements of low power
consumption for most of the battery powered real
time visual communication applications and
reduced computational complexity, new fast
algorithms have been developed. They reduce the
search area at the cost of visual quality
however the visual quality is not impaired beyond
certain threshold. The various fast motion
estimation algorithms can be classified as 10
• -Search area subsampling
• a. fixed-search area
• TSS,OTS,NTSS,4SS,OSA,Cross Search, 2DLOG
• zone based search, spiral based search
• -Hierarchical/Multiresolution
• Subsampling, HPDS, Gaussian, reduced pel
• -Pel Decimation
• 21, 41, 161
• -Prediction based
• Temporal, Spatial, Linear, Non-Linear

17
Comparative study of Motion Estimation (ME)
Algorithms
• Three step search
• Three step search (TSS) 5, employs rectangular
search patterns with different sizes.
• Koga et al 5 introduced this algorithm in 1981.
• It became very popular because of its simplicity
and also robust and near optimal performance.
• It searches for the best motion vectors in a
coarse to fine search pattern.
• One problem that occurs with the three step
search is that it uses a uniformly allocated
checking point pattern in the first step, which
becomes inefficient for small motion estimation.

18
Comparative study of Motion Estimation (ME)
Algorithms
• Experimental results 5 show that the block
motion field of a real world image sequence is
usually gentle, smooth and varies slowly. This
results in center-biased global minimum
• So, the checking point pattern should also be
center- biased.

19
Comparative study of Motion Estimation (ME)
Algorithms
• New three step search (NTSS)
• To remedy the problem with TSS, NTSS was
proposed.
• It employs a center-biased checking point
pattern in the first step, which is derived by
making the search adaptive to the motion vector
distribution, and a halfway-stop technique to
reduce the computation cost.
• Fig 7-a 5 and Fig 7-b 5 explain the
implementation.
• In Fig 7-a, filled circles are the checking
points in the first step of TSS, squares are the
8 extra points added in the first step of NTSS,
and triangles explain how the second step search
is performed if the minimum BDM (block distortion
measure) in the first step is at one of the 8
neighbors of the window center.
• Fig 7-b shows the block diagram of NTSS
algorithm.
• Fig 7 Implementation details of NTSS Algorithm
5

20
Comparative study of Motion Estimation (ME)
Algorithms
• Four step search
• This algorithm also exploits the center-biased
property in its search.
• The computational complexity of the four-step
search is less than that of the new three step
search, while the performance in terms of quality
is as good.
• It is also more robust than the three step search
and it maintains its performance for image
sequences with complex movements like camera
zooming and fast motion. Hence it is a very
attractive strategy for motion estimation.
• Fig. 8 Search patterns of the 4SS (a) First
step, (b) second/third step (c)second/third step
(d) fourth step 13

21
Comparative study of Motion Estimation (ME)
Algorithms
• Search points needed (considering the
displacement parameter p in -7,7 range)-
• Minimum 98 17
• Worst case 9558 27 (Large motion)
• Thus computational complexity is just two block
matches more compared to TSS and lesser compared
to 33 block matches in NTSS.
• Performance comparable to NTSS.

Fig. 9
22
Comparative study of Motion Estimation (ME)
Algorithms
• 2-D Logarithmic search
• Although 2-D logarithmic algorithms require more
steps than the three step search, it can be more
accurate, especially when the search window is
large6.
• This procedure requires only searching 13 to 21
locations.
• Displacement parameter p 5

So n 2 for p5
23
Comparative study of Motion Estimation (ME)
Algorithms
• One-at-a-time search (OTS)
• One-at-a-time search (OTS) is a basic
two-variable optimization search method.
• Each variable is adjusted with the other fixed.
• Direction of search in each step will be parallel
to one of the coordinate axes.
• Fig. 11 11 illustrates the OTS for a convex
function in two variables.
• Fig 11 Illustration of one at a time search 11

24
Comparative study of Motion Estimation (ME)
Algorithms
• Conjugate direction search (CDS)
• An extension of the one-at-a-time search is the
conjugate direction search (CDS) as shown in Fig
12 11.
• The directions of search, rather than being kept
fixed, are changed to improve the search
efficiency.
• The CDS obtains n linearly independent conjugate
direction vectors, n being the number of
variables in the objective function.
• A succession of one-at-a-time searches in each of
n sets of independent directions starting with
the coordinate directions is carried out.

25
Comparative study of Motion Estimation (ME)
Algorithms
• Diamond search (DS)
• As indicated in Table 1, about 52.76 to 96.09
of the motion vectors are enclosed in a circular
support with a radium of 2 pels and centered on
the zero-motion position.
• TSS uses a sparse searching point pattern. This
could yield the search path in wrong direction
and thus give a wrong optimum point.
• Other methods which use a smaller search pattern
can cause the MDB to be trapped to a local
minimum.

Table 1 Motion vector distribution aggregately
measured at various motion distances (in pel)
with regard to the center position using the Full
Search algorithm based on MSE matching criterion.
15
26
Comparative study of Motion Estimation (ME)
Algorithms
• Diamond search (DS) (contd..)
• Fig 14 An appropriate search pattern
supportcircular area with a radium of 2 pels.
The 13 crosses show all possible checking points
within the circle 15

27
Comparative study of Motion Estimation (ME)
Algorithms
• The DS algorithm employs two search patterns as
illustrated in Fig. 15, which are derived from
the crosses () in Fig. 14.
• The first pattern, called large diamond search
pattern (LDSP), comprises nine checking points
from which eight points surround the center one
to compose a diamond shape.
• The second pattern consisting of five checking
points forms a smaller diamond shape, called
small diamond search pattern (SDSP).
• In the searching procedure of the DS algorithm,
LDSP is repeatedly used until the step in which
the minimum block distortion (MBD) occurs at the
center point.
• The search pattern is then switched from LDSP to
SDSP as reaching to the final search stage.
• Among the five checking points in SDSP, the
position yielding the MBD provides the motion
vector of the best matching block.

28
Comparative study of Motion Estimation (ME)
Algorithms
Fig. 17 Diamond search path example which leads
to the motion vector (-4, -2) in five search
steps. There are 24 search points in total -
taking 9, 5, 3, 3 and 4 search points in each
step sequentially. 15
29
Comparative study of Motion Estimation (ME)
Algorithms
• Feature-matching
• Feature-matching algorithms can be seen as
matching in meta information extracted from the
current block and search area pels as shown in
Fig.18 10
• In most of the feature matching algorithms, the
meta information has a higher entropy than the
original information, thus reducing the
arithmetic computational load, as well as the
memory bandwidth within the matching process.
• The feature extraction is performed by
morphological filters, reduced pel resolution or
projection methods.
• Integral projection matching (IPM), projection
three step search (PTSS), successive elimination
algorithm (SEA), binary block matching (BBM), bit
plane matching (BPM), block feature matching
(BFM) are some of the algorithms based on feature
matching.

30
Comparative study of Motion Estimation (ME)
Algorithms
• 3. Fixed size block motion estimation (FSBME) and
variable size block motion estimation (VSBME) 3
• The advantage of the former is that there is no
need to provide segmentation information. However
it is not very efficient when we have images that
have both really fast and slow motions.
• In this case the latter method is more useful
because we can segment the image into smaller
blocks in areas where there is complex motion and
larger blocks when there is very little motion.

31
Comparative study of Motion Estimation (ME)
Algorithms
• 3. Simulation
• The simulation results for exhaustive search,
TSS, NTSS, 4SS, DS on different test sequences
have been tabulated.
• Cronkite (256X256), toy vehicle (512X512),
chemical plant (256X256) and caltrain test
sequences 19 were used in the simulation. These
are obtained from the digitization of 24 frames
per second motion pictures.
• The source code for the various algorithms is
available from Matlab central file exchange 20.
This code was modified for different images.
• The distance criterion method used is MAE.
• Block size of 8X8 (16X16) is used. Search window
size is (p p block size) X (p p block
size) where p (7) is the displacement parameter.
• The various algorithms are compared based on
their computational complexity and PSNR.

32
Comparative study of Motion Estimation (ME)
Algorithms
Table 2 Average number of computations needed
for different ME algorithms
33
Comparative study of Motion Estimation (ME)
Algorithms
Table 3 Average PSNR (dB) for different ME
algorithms
34
Comparative study of Motion Estimation (ME)
Algorithms
• Simulation results show that ES gives the best
performance (highest PSNR) at the cost of high
computational complexity. Motion estimation
constitutes about 60 of the total computational
load in encoders, so exhaustive search algorithm
is not the most feasible technique to employ
motion estimation.
• TSS algorithm gives good performance but it can
not always detect small motions.
• NTSS is good for slow motions but its
computational complexity is higher than that of
TSS for large motion.
• 4SS has a small initial step and is good to
detect small motion. Its computational complexity
is comparable to that of TSS and lower than that
of NTSS.
• DS performs well both computationally and in
terms of quality.

35
Comparative study of Motion Estimation (ME)
Algorithms
• References
• 1 Z. Wei et al Efficient fast motion
estimation algorithm for H.264 based on
polynomial model, IMACS Multiconference on
computational engineering in System Applications,
pp. 1654-1657, Oct. 4-6, 2006.
• 2 T. Huang, Y. Hsu and R. Tsai Interframe
coding with general two-dimensional motion
compensation, IEEE International Conference on
ICASSP 82 (Acoustics, Speech and Signal
Processing), vol. 7, pp. 464 - 466, May 1982
•
• 3 M. Manikandan, P. Vijayakumar and N.
Ramadass, Motion estimation method for video
compression - an overview, Wireless and Optical
Communications Networks, IFIP Intl Conference,
pp. 5-11, 13 April 2006.
• 4 H. Ferhatosmanoglu et al, Approximate
nearest neighbor searching in multimedia
databases, Data Engineering, 2001. Proceedings.
17th International Conference on, pp. 503 511,
2-6 April 2001.
• 5 R. Li, B. Zeng, and M. L. Liou, A new
three-step search algorithm for block motion
estimation IEEE Transactions on Circuits and
Systems Video Technology,vol. 4, pp. 438443,
Aug. 1994.
• 6 J. R. Jain and A. K. Jain, Displacement
measurement and its application in interframe
image coding IEEE Transactions on
Communications, vol. 29, pp. 1799-1808, Dec.
1981.

36
Comparative study of Motion Estimation (ME)
Algorithms
• 7 J. Kim and S. Yang, An efficient search
algorithm for block motion estimation IEEE
Workshop on Signal Processing Systems, pp.
100-109, Oct. 1999.
•
• 8 C. Zhu et al., Enhanced hexagonal search for
fast block motion estimation, IEEE Transactions
on Circuits and Systems for Video Technology,
vol. 14, pp. 1210 - 1214, Oct. 2004.
•
• 9 E. Fernandez et al., Reducing motion
estimation complexity in MPEG-2 to H.264
transcoding, IEEE International Conference on
Multimedia and Expo, 2007 ,pp. 440-443, 2-5 July
2007.
•
• 10 K. Peter, Algorithms,complexity analysis
and vlsi architectures for MPEG-4 motion
1999.
•
• 11 R. Srinivasan and K. R. Rao, Predictive
coding based on efficient motion estimation IEEE
Transactions on Communications , vol. 33, pp.
888-896, Aug. 1985.
• 12 I. Amre et al, An efficient variable block
size selection scheme for the H.264 motion
estimation 6th International Workshop on
System-On-Chip for Real-Time application, pp.
5-9, Dec. 2006.

37
Comparative study of Motion Estimation (ME)
Algorithms
• 13 P. Lai-Man and M. Wing-chung, A novel
four-step search algorithm for fast
blockmatching, IEEE Transactions On Circuits and
Systems for Video Technology, vol. 6, pp.
313-317, Jun. 1996.
•
• 14 H. Zhongli and M.L. Lieo, A high
performance fast search algorithm for block
matching motion estimation, IEEE Transactions on
Circuits and Systems for Video Technology, vol.
7, pp. 826-828, Oct. 1997.
• 15 S. Zhu and K. K. Ma A new diamond search
algorithm for fast block-matching motion
estimation, IEEE Transactions on Image
Processing, vol. 9, pp. 287-290, Feb. 2000.
• 16 Y. Chan and W. Siu , An efficient search
strategy for block motion estimation using image
features, IEEE Transactions on Image Processing,
vol. 10, pp. 1223-1238, Aug. 2001.
• 17 J. Lee et al, An efficient architecture for
motion estimation and compensation in the
transform domain, IEEE Transactions on Circuits
and Systems for Video Technology, vol. 16, pp.
191-201,Feb. 2006.
• 18 C. Cheung and L. Po, A hierarchical block
motion estimation algorithm using partial
distortion measure, Proceedings International
Conference on Image Processing, vol. 3, pp.
606-609, Oct. 1997
• 19 Test sequences obtained from
• http//sipi.usc.edu/database/database.cgi
?volumesequences
•
• 20 Motion estimation algorithm source code
available at Matlab central file exchange
• www.mathworks.com/matlabcentral/fileexcha
nge
• 21 Phase correlation motion estimation
report available at http//visilab.unime.it/iann
i/slides_CV/liang_report-phase-correlation-motion-
prediction.pdf
•

38
Comparative study of Motion Estimation (ME)
Algorithms
• APPENDIX A
• ACRONYMS
• 2DLOG 2-d Logarithmic Search
• BBM Binary Block Matching
• BDM Block Distortion Measure
• BFM Block Feature Matching
• BPM Bit Plane Matching
• DCT Discrete Cosine Transform
• DS Diamond Search
• FS Full Search
• FSBME Fixed Size Block Motion Estimation
• H.264/AVC Video Coding Standard
• HPDS Hierarchical Partial Distortion Search
• IPM Integral Projection Matching
• MAE Mean Absolute Error
• MB Macro Block
• ME Motion Estimation
• MSE Mean Square Error
• MV Motion Vector

39
Comparative study of Motion Estimation (ME)
Algorithms
• Questions?