Loading...

PPT – EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms PowerPoint presentation | free to download - id: 247b6c-M2JiO

The Adobe Flash plugin is needed to view this content

EE5359 Spring 2008 Comparative study of Motion

Estimation (ME) Algorithms

- Khyati Mistry
- 1000552796
- 04/29/2008

Comparative study of Motion Estimation (ME)

Algorithms

- Objective
- The goal of this project is to do a comparative

study of the different motion estimation

techniques and search algorithms that have been

proposed for interframe coding in moving video

sequences taking into consideration the different

kinds of motion of the moving objects in

successive frames translational, rotational,

zooming etc

Comparative study of Motion Estimation (ME)

Algorithms

- Introduction
- The motion estimation module will create a

model for the current frame by modifying the

reference frames such that it is a very close

match to the current frame. This estimated

current frame is then motion compensated and the

compensated residual image is then encoded and

transmitted.

Fig 1 Motion estimation block diagram 1

Comparative study of Motion Estimation (ME)

Algorithms

- The objective is to estimate the motion vectors

from two time sequential frames of the image. - The objective is to estimate the motion vectors

from two time sequential frames of the image. The

motion of an object in the 3-d object space is

translated into two successive frames in the

image space at time instants t1 and t2 as shown

in Fig 2.

- Where
- t1, t2 represent the time axis such that t2 gt

t1 - (X, Y) Image-space coordinates of P in the

scene at time t1 - (X, Y) Image space coordinates of P at time

t2 gt t1 - (x, y, z) Object-space coordinates at a point P

in the scene at time t1

Fig 2Basic geometry for motion estimation 2

Comparative study of Motion Estimation (ME)

Algorithms

- Broad Classification of Motion Estimation

Algorithms

Fig 3. Motion estimation algorithms 10

Comparative study of Motion Estimation (ME)

Algorithms

- Frequency domain algorithms In these algorithms,

the algorithm is applied on the transformed

coefficients 10. The different

algorithms/techniques used are phase correlation

21, matching in wavelet and DCT domain 17. - Time domain algorithms These comprise of

matching algorithms and gradient-based algorithms

1-16. - Block matching algorithms match all or some

pixels in the current block with the block in the

search area of reference frame based on some cost

function 1-15. - Feature based algorithms match the meta

information/data of the current block with that

of the block in reference frame 16.

Comparative study of Motion Estimation (ME)

Algorithms

Time domain Vs frequency domain algorithms

Fig 4 (a) Conventional video encoder with

motion estimation and compensation in the spatial

domain. (b) Simplified video encoder with motion

estimation and compensation in the transform

domain. 17

Comparative study of Motion Estimation (ME)

Algorithms

- In the frequency domain, the phase correlation

algorithm provides accurate predictions but is

based on the fast Fourier transform (FFT), which

is incompatible with current DCT-based video

coding standards and which has high computational

complexity since a large search window is

necessary. Also, it gives poor performance when

block size is small and motion is not pure

translational. - DCT-based motion estimation simplifies the

conventional DCT-based video coders achieving

spatial redundancy reduction through DCT and

temporal redundancy reduction through motion

estimation and compensation. - In the conventional DCT-based video coder, the

feedback loop has the following functional

blocks DCT, inverse DCT (IDCT), quantizer,

inverse quantizer, and spatial domain motion

estimation and compensation. - If motion estimation and compensation are

performed entirely in the DCT domain, IDCT can be

removed from the feedback loop. Therefore, the

feedback loop contains the reduced functional

blocks quantizer, inverse quantizer, and

transform domain motion estimation and

compensation (Fig 4) - High computational complexity is still the main

drawback of the DCT-based motion estimation and

compensation approach.

Comparative study of Motion Estimation (ME)

Algorithms

- Major Aspects of motion estimation techniques
- Reduced computational complexity
- Good visual quality in terms of representation

of true motion - High compression ratio

Comparative study of Motion Estimation (ME)

Algorithms

- The computational complexity of a motion

estimation technique can be determined by three

factors - The search algorithm decides the overall

computational complexity and accuracy of motion

estimation - Search area the span of search window to find

the best match - Cost function the different cost metrics used to

find the best match

Comparative study of Motion Estimation (ME)

Algorithms

- Cost Functions
- The different cost functions that are used to

find the best possible match are listed 10 - CCF (cross correlation function)
- CCF(dx,dy)
- (MVx, MVy) max (dx,dy)?R2 CCF(dx,dy)

Comparative study of Motion Estimation (ME)

Algorithms

- MSE (mean square error)
- MSE(dx,dy)
- (MVx, MVy) min (dx,dy)?R2 MSE(dx,dy)

Comparative study of Motion Estimation (ME)

Algorithms

- MAE (mean absolute error)
- MAE(dx,dy)
- (MVx, MVy) min (dx,dy)?R2 MAE(dx,dy)

Comparative study of Motion Estimation (ME)

Algorithms

- SAD (sum of absolute differences)
- SAD(dx,dy)
- (MVx, MVy) min (dx,dy)?R2 SAD(dx,dy)

Comparative study of Motion Estimation (ME)

Algorithms

- Motion estimation Algorithms
- 1. Full search
- This is the brute force technique in which each

block is matched with the current block to find

the best possible match. - It is a very simple and accurate technique.
- Not efficient because it involves high

computational cost. - So more efficient algorithms were proposed but

there was always a tradeoff between efficiency

and image quality.

- Fig 5 Full search (Brute force

technique)

Comparative study of Motion Estimation (ME)

Algorithms

- 2. Fast motion estimation algorithms
- To meet the requirements of low power

consumption for most of the battery powered real

time visual communication applications and

reduced computational complexity, new fast

algorithms have been developed. They reduce the

search area at the cost of visual quality

however the visual quality is not impaired beyond

certain threshold. The various fast motion

estimation algorithms can be classified as 10 - -Search area subsampling
- a. fixed-search area
- TSS,OTS,NTSS,4SS,OSA,Cross Search, 2DLOG
- b. adaptive search area
- zone based search, spiral based search
- -Hierarchical/Multiresolution
- Subsampling, HPDS, Gaussian, reduced pel
- -Pel Decimation
- 21, 41, 161
- -Prediction based
- Temporal, Spatial, Linear, Non-Linear

Comparative study of Motion Estimation (ME)

Algorithms

- Three step search
- Three step search (TSS) 5, employs rectangular

search patterns with different sizes. - Koga et al 5 introduced this algorithm in 1981.
- It became very popular because of its simplicity

and also robust and near optimal performance. - It searches for the best motion vectors in a

coarse to fine search pattern. - One problem that occurs with the three step

search is that it uses a uniformly allocated

checking point pattern in the first step, which

becomes inefficient for small motion estimation.

Comparative study of Motion Estimation (ME)

Algorithms

- Experimental results 5 show that the block

motion field of a real world image sequence is

usually gentle, smooth and varies slowly. This

results in center-biased global minimum

distribution instead of uniform distribution. - So, the checking point pattern should also be

center- biased.

Comparative study of Motion Estimation (ME)

Algorithms

- New three step search (NTSS)
- To remedy the problem with TSS, NTSS was

proposed. - It employs a center-biased checking point

pattern in the first step, which is derived by

making the search adaptive to the motion vector

distribution, and a halfway-stop technique to

reduce the computation cost. - Fig 7-a 5 and Fig 7-b 5 explain the

implementation. - In Fig 7-a, filled circles are the checking

points in the first step of TSS, squares are the

8 extra points added in the first step of NTSS,

and triangles explain how the second step search

is performed if the minimum BDM (block distortion

measure) in the first step is at one of the 8

neighbors of the window center. - Fig 7-b shows the block diagram of NTSS

algorithm.

- Fig 7 Implementation details of NTSS Algorithm

5

Comparative study of Motion Estimation (ME)

Algorithms

- Four step search
- This algorithm also exploits the center-biased

property in its search. - The computational complexity of the four-step

search is less than that of the new three step

search, while the performance in terms of quality

is as good. - It is also more robust than the three step search

and it maintains its performance for image

sequences with complex movements like camera

zooming and fast motion. Hence it is a very

attractive strategy for motion estimation.

- Fig. 8 Search patterns of the 4SS (a) First

step, (b) second/third step (c)second/third step

(d) fourth step 13

Comparative study of Motion Estimation (ME)

Algorithms

- Search points needed (considering the

displacement parameter p in -7,7 range)- - Minimum 98 17
- Worst case 9558 27 (Large motion)
- Thus computational complexity is just two block

matches more compared to TSS and lesser compared

to 33 block matches in NTSS. - Performance comparable to NTSS.

Fig. 9

Comparative study of Motion Estimation (ME)

Algorithms

- 2-D Logarithmic search
- Although 2-D logarithmic algorithms require more

steps than the three step search, it can be more

accurate, especially when the search window is

large6. - This procedure requires only searching 13 to 21

locations. - Displacement parameter p 5

So n 2 for p5

Comparative study of Motion Estimation (ME)

Algorithms

- One-at-a-time search (OTS)
- One-at-a-time search (OTS) is a basic

two-variable optimization search method. - Each variable is adjusted with the other fixed.
- Direction of search in each step will be parallel

to one of the coordinate axes. - Fig. 11 11 illustrates the OTS for a convex

function in two variables.

- Fig 11 Illustration of one at a time search 11

Comparative study of Motion Estimation (ME)

Algorithms

- Conjugate direction search (CDS)
- An extension of the one-at-a-time search is the

conjugate direction search (CDS) as shown in Fig

12 11. - The directions of search, rather than being kept

fixed, are changed to improve the search

efficiency. - The CDS obtains n linearly independent conjugate

direction vectors, n being the number of

variables in the objective function. - A succession of one-at-a-time searches in each of

n sets of independent directions starting with

the coordinate directions is carried out.

Comparative study of Motion Estimation (ME)

Algorithms

- Diamond search (DS)

- As indicated in Table 1, about 52.76 to 96.09

of the motion vectors are enclosed in a circular

support with a radium of 2 pels and centered on

the zero-motion position. - TSS uses a sparse searching point pattern. This

could yield the search path in wrong direction

and thus give a wrong optimum point. - Other methods which use a smaller search pattern

can cause the MDB to be trapped to a local

minimum.

Table 1 Motion vector distribution aggregately

measured at various motion distances (in pel)

with regard to the center position using the Full

Search algorithm based on MSE matching criterion.

15

Comparative study of Motion Estimation (ME)

Algorithms

- Diamond search (DS) (contd..)

- Fig 14 An appropriate search pattern

supportcircular area with a radium of 2 pels.

The 13 crosses show all possible checking points

within the circle 15

Comparative study of Motion Estimation (ME)

Algorithms

- The DS algorithm employs two search patterns as

illustrated in Fig. 15, which are derived from

the crosses () in Fig. 14. - The first pattern, called large diamond search

pattern (LDSP), comprises nine checking points

from which eight points surround the center one

to compose a diamond shape. - The second pattern consisting of five checking

points forms a smaller diamond shape, called

small diamond search pattern (SDSP). - In the searching procedure of the DS algorithm,

LDSP is repeatedly used until the step in which

the minimum block distortion (MBD) occurs at the

center point. - The search pattern is then switched from LDSP to

SDSP as reaching to the final search stage. - Among the five checking points in SDSP, the

position yielding the MBD provides the motion

vector of the best matching block.

Comparative study of Motion Estimation (ME)

Algorithms

Fig. 17 Diamond search path example which leads

to the motion vector (-4, -2) in five search

steps. There are 24 search points in total -

taking 9, 5, 3, 3 and 4 search points in each

step sequentially. 15

Comparative study of Motion Estimation (ME)

Algorithms

- Feature-matching
- Feature-matching algorithms can be seen as

matching in meta information extracted from the

current block and search area pels as shown in

Fig.18 10 - In most of the feature matching algorithms, the

meta information has a higher entropy than the

original information, thus reducing the

arithmetic computational load, as well as the

memory bandwidth within the matching process. - The feature extraction is performed by

morphological filters, reduced pel resolution or

projection methods. - Integral projection matching (IPM), projection

three step search (PTSS), successive elimination

algorithm (SEA), binary block matching (BBM), bit

plane matching (BPM), block feature matching

(BFM) are some of the algorithms based on feature

matching.

Comparative study of Motion Estimation (ME)

Algorithms

- 3. Fixed size block motion estimation (FSBME) and

variable size block motion estimation (VSBME) 3 - The advantage of the former is that there is no

need to provide segmentation information. However

it is not very efficient when we have images that

have both really fast and slow motions. - In this case the latter method is more useful

because we can segment the image into smaller

blocks in areas where there is complex motion and

larger blocks when there is very little motion.

Comparative study of Motion Estimation (ME)

Algorithms

- 3. Simulation
- The simulation results for exhaustive search,

TSS, NTSS, 4SS, DS on different test sequences

have been tabulated. - Cronkite (256X256), toy vehicle (512X512),

chemical plant (256X256) and caltrain test

sequences 19 were used in the simulation. These

are obtained from the digitization of 24 frames

per second motion pictures. - The source code for the various algorithms is

available from Matlab central file exchange 20.

This code was modified for different images. - The distance criterion method used is MAE.
- Block size of 8X8 (16X16) is used. Search window

size is (p p block size) X (p p block

size) where p (7) is the displacement parameter. - The various algorithms are compared based on

their computational complexity and PSNR.

Comparative study of Motion Estimation (ME)

Algorithms

Table 2 Average number of computations needed

for different ME algorithms

Comparative study of Motion Estimation (ME)

Algorithms

Table 3 Average PSNR (dB) for different ME

algorithms

Comparative study of Motion Estimation (ME)

Algorithms

- Simulation results show that ES gives the best

performance (highest PSNR) at the cost of high

computational complexity. Motion estimation

constitutes about 60 of the total computational

load in encoders, so exhaustive search algorithm

is not the most feasible technique to employ

motion estimation. - TSS algorithm gives good performance but it can

not always detect small motions. - NTSS is good for slow motions but its

computational complexity is higher than that of

TSS for large motion. - 4SS has a small initial step and is good to

detect small motion. Its computational complexity

is comparable to that of TSS and lower than that

of NTSS. - DS performs well both computationally and in

terms of quality.

Comparative study of Motion Estimation (ME)

Algorithms

- References
- 1 Z. Wei et al Efficient fast motion

estimation algorithm for H.264 based on

polynomial model, IMACS Multiconference on

computational engineering in System Applications,

pp. 1654-1657, Oct. 4-6, 2006. - 2 T. Huang, Y. Hsu and R. Tsai Interframe

coding with general two-dimensional motion

compensation, IEEE International Conference on

ICASSP 82 (Acoustics, Speech and Signal

Processing), vol. 7, pp. 464 - 466, May 1982 - 3 M. Manikandan, P. Vijayakumar and N.

Ramadass, Motion estimation method for video

compression - an overview, Wireless and Optical

Communications Networks, IFIP Intl Conference,

pp. 5-11, 13 April 2006. - 4 H. Ferhatosmanoglu et al, Approximate

nearest neighbor searching in multimedia

databases, Data Engineering, 2001. Proceedings.

17th International Conference on, pp. 503 511,

2-6 April 2001. - 5 R. Li, B. Zeng, and M. L. Liou, A new

three-step search algorithm for block motion

estimation IEEE Transactions on Circuits and

Systems Video Technology,vol. 4, pp. 438443,

Aug. 1994. - 6 J. R. Jain and A. K. Jain, Displacement

measurement and its application in interframe

image coding IEEE Transactions on

Communications, vol. 29, pp. 1799-1808, Dec.

1981.

Comparative study of Motion Estimation (ME)

Algorithms

- 7 J. Kim and S. Yang, An efficient search

algorithm for block motion estimation IEEE

Workshop on Signal Processing Systems, pp.

100-109, Oct. 1999. - 8 C. Zhu et al., Enhanced hexagonal search for

fast block motion estimation, IEEE Transactions

on Circuits and Systems for Video Technology,

vol. 14, pp. 1210 - 1214, Oct. 2004. - 9 E. Fernandez et al., Reducing motion

estimation complexity in MPEG-2 to H.264

transcoding, IEEE International Conference on

Multimedia and Expo, 2007 ,pp. 440-443, 2-5 July

2007. - 10 K. Peter, Algorithms,complexity analysis

and vlsi architectures for MPEG-4 motion

estimation, Kluwer Academic Publishers, Boston,

1999. - 11 R. Srinivasan and K. R. Rao, Predictive

coding based on efficient motion estimation IEEE

Transactions on Communications , vol. 33, pp.

888-896, Aug. 1985. - 12 I. Amre et al, An efficient variable block

size selection scheme for the H.264 motion

estimation 6th International Workshop on

System-On-Chip for Real-Time application, pp.

5-9, Dec. 2006.

Comparative study of Motion Estimation (ME)

Algorithms

- 13 P. Lai-Man and M. Wing-chung, A novel

four-step search algorithm for fast

blockmatching, IEEE Transactions On Circuits and

Systems for Video Technology, vol. 6, pp.

313-317, Jun. 1996. - 14 H. Zhongli and M.L. Lieo, A high

performance fast search algorithm for block

matching motion estimation, IEEE Transactions on

Circuits and Systems for Video Technology, vol.

7, pp. 826-828, Oct. 1997. - 15 S. Zhu and K. K. Ma A new diamond search

algorithm for fast block-matching motion

estimation, IEEE Transactions on Image

Processing, vol. 9, pp. 287-290, Feb. 2000. - 16 Y. Chan and W. Siu , An efficient search

strategy for block motion estimation using image

features, IEEE Transactions on Image Processing,

vol. 10, pp. 1223-1238, Aug. 2001. - 17 J. Lee et al, An efficient architecture for

motion estimation and compensation in the

transform domain, IEEE Transactions on Circuits

and Systems for Video Technology, vol. 16, pp.

191-201,Feb. 2006. - 18 C. Cheung and L. Po, A hierarchical block

motion estimation algorithm using partial

distortion measure, Proceedings International

Conference on Image Processing, vol. 3, pp.

606-609, Oct. 1997 - 19 Test sequences obtained from
- http//sipi.usc.edu/database/database.cgi

?volumesequences - 20 Motion estimation algorithm source code

available at Matlab central file exchange - www.mathworks.com/matlabcentral/fileexcha

nge - 21 Phase correlation motion estimation

report available at http//visilab.unime.it/iann

i/slides_CV/liang_report-phase-correlation-motion-

prediction.pdf

Comparative study of Motion Estimation (ME)

Algorithms

- APPENDIX A
- ACRONYMS
- 2DLOG 2-d Logarithmic Search
- BBM Binary Block Matching
- BDM Block Distortion Measure
- BFM Block Feature Matching
- BPM Bit Plane Matching
- DCT Discrete Cosine Transform
- DS Diamond Search
- FS Full Search
- FSBME Fixed Size Block Motion Estimation
- H.264/AVC Video Coding Standard
- HPDS Hierarchical Partial Distortion Search
- IPM Integral Projection Matching
- MAE Mean Absolute Error
- MB Macro Block
- ME Motion Estimation
- MSE Mean Square Error
- MV Motion Vector

Comparative study of Motion Estimation (ME)

Algorithms

- Questions?