EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms - PowerPoint PPT Presentation

Loading...

PPT – EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms PowerPoint presentation | free to download - id: 247b6c-M2JiO



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms

Description:

The objective is to estimate the motion vectors from two time sequential frames of the image. ... complexity is still the main drawback of the DCT-based motion ... – PowerPoint PPT presentation

Number of Views:110
Avg rating:3.0/5.0
Slides: 40
Provided by: Qual69
Learn more at: http://www-ee.uta.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: EE5359 Spring 2008 Comparative study of Motion Estimation ME Algorithms


1
EE5359 Spring 2008 Comparative study of Motion
Estimation (ME) Algorithms
  • Khyati Mistry
  • 1000552796
  • 04/29/2008

2
Comparative study of Motion Estimation (ME)
Algorithms
  • Objective
  • The goal of this project is to do a comparative
    study of the different motion estimation
    techniques and search algorithms that have been
    proposed for interframe coding in moving video
    sequences taking into consideration the different
    kinds of motion of the moving objects in
    successive frames translational, rotational,
    zooming etc

3
Comparative study of Motion Estimation (ME)
Algorithms
  • Introduction
  • The motion estimation module will create a
    model for the current frame by modifying the
    reference frames such that it is a very close
    match to the current frame. This estimated
    current frame is then motion compensated and the
    compensated residual image is then encoded and
    transmitted.

Fig 1 Motion estimation block diagram 1
4
Comparative study of Motion Estimation (ME)
Algorithms
  • The objective is to estimate the motion vectors
    from two time sequential frames of the image.
  • The objective is to estimate the motion vectors
    from two time sequential frames of the image. The
    motion of an object in the 3-d object space is
    translated into two successive frames in the
    image space at time instants t1 and t2 as shown
    in Fig 2.
  • Where
  • t1, t2 represent the time axis such that t2 gt
    t1
  • (X, Y) Image-space coordinates of P in the
    scene at time t1
  • (X, Y) Image space coordinates of P at time
    t2 gt t1
  • (x, y, z) Object-space coordinates at a point P
    in the scene at time t1

Fig 2Basic geometry for motion estimation 2
5
Comparative study of Motion Estimation (ME)
Algorithms
  • Broad Classification of Motion Estimation
    Algorithms

Fig 3. Motion estimation algorithms 10
6
Comparative study of Motion Estimation (ME)
Algorithms
  • Frequency domain algorithms In these algorithms,
    the algorithm is applied on the transformed
    coefficients 10. The different
    algorithms/techniques used are phase correlation
    21, matching in wavelet and DCT domain 17.
  • Time domain algorithms These comprise of
    matching algorithms and gradient-based algorithms
    1-16.
  • Block matching algorithms match all or some
    pixels in the current block with the block in the
    search area of reference frame based on some cost
    function 1-15.
  • Feature based algorithms match the meta
    information/data of the current block with that
    of the block in reference frame 16.

7
Comparative study of Motion Estimation (ME)
Algorithms
Time domain Vs frequency domain algorithms
Fig 4 (a) Conventional video encoder with
motion estimation and compensation in the spatial
domain. (b) Simplified video encoder with motion
estimation and compensation in the transform
domain. 17
8
Comparative study of Motion Estimation (ME)
Algorithms
  • In the frequency domain, the phase correlation
    algorithm provides accurate predictions but is
    based on the fast Fourier transform (FFT), which
    is incompatible with current DCT-based video
    coding standards and which has high computational
    complexity since a large search window is
    necessary. Also, it gives poor performance when
    block size is small and motion is not pure
    translational.
  • DCT-based motion estimation simplifies the
    conventional DCT-based video coders achieving
    spatial redundancy reduction through DCT and
    temporal redundancy reduction through motion
    estimation and compensation.
  • In the conventional DCT-based video coder, the
    feedback loop has the following functional
    blocks DCT, inverse DCT (IDCT), quantizer,
    inverse quantizer, and spatial domain motion
    estimation and compensation.
  • If motion estimation and compensation are
    performed entirely in the DCT domain, IDCT can be
    removed from the feedback loop. Therefore, the
    feedback loop contains the reduced functional
    blocks quantizer, inverse quantizer, and
    transform domain motion estimation and
    compensation (Fig 4)
  • High computational complexity is still the main
    drawback of the DCT-based motion estimation and
    compensation approach.

9
Comparative study of Motion Estimation (ME)
Algorithms
  • Major Aspects of motion estimation techniques
  • Reduced computational complexity
  • Good visual quality in terms of representation
    of true motion
  • High compression ratio

10
Comparative study of Motion Estimation (ME)
Algorithms
  • The computational complexity of a motion
    estimation technique can be determined by three
    factors
  • The search algorithm decides the overall
    computational complexity and accuracy of motion
    estimation
  • Search area the span of search window to find
    the best match
  • Cost function the different cost metrics used to
    find the best match

11
Comparative study of Motion Estimation (ME)
Algorithms
  • Cost Functions
  • The different cost functions that are used to
    find the best possible match are listed 10
  • CCF (cross correlation function)
  • CCF(dx,dy)
  • (MVx, MVy) max (dx,dy)?R2 CCF(dx,dy)

12
Comparative study of Motion Estimation (ME)
Algorithms
  • MSE (mean square error)
  • MSE(dx,dy)
  • (MVx, MVy) min (dx,dy)?R2 MSE(dx,dy)

13
Comparative study of Motion Estimation (ME)
Algorithms
  • MAE (mean absolute error)
  • MAE(dx,dy)
  • (MVx, MVy) min (dx,dy)?R2 MAE(dx,dy)

14
Comparative study of Motion Estimation (ME)
Algorithms
  • SAD (sum of absolute differences)
  • SAD(dx,dy)
  • (MVx, MVy) min (dx,dy)?R2 SAD(dx,dy)

15
Comparative study of Motion Estimation (ME)
Algorithms
  • Motion estimation Algorithms
  • 1. Full search
  • This is the brute force technique in which each
    block is matched with the current block to find
    the best possible match.
  • It is a very simple and accurate technique.
  • Not efficient because it involves high
    computational cost.
  • So more efficient algorithms were proposed but
    there was always a tradeoff between efficiency
    and image quality.
  • Fig 5 Full search (Brute force
    technique)

16
Comparative study of Motion Estimation (ME)
Algorithms
  • 2. Fast motion estimation algorithms
  • To meet the requirements of low power
    consumption for most of the battery powered real
    time visual communication applications and
    reduced computational complexity, new fast
    algorithms have been developed. They reduce the
    search area at the cost of visual quality
    however the visual quality is not impaired beyond
    certain threshold. The various fast motion
    estimation algorithms can be classified as 10
  • -Search area subsampling
  • a. fixed-search area
  • TSS,OTS,NTSS,4SS,OSA,Cross Search, 2DLOG
  • b. adaptive search area
  • zone based search, spiral based search
  • -Hierarchical/Multiresolution
  • Subsampling, HPDS, Gaussian, reduced pel
  • -Pel Decimation
  • 21, 41, 161
  • -Prediction based
  • Temporal, Spatial, Linear, Non-Linear

17
Comparative study of Motion Estimation (ME)
Algorithms
  • Three step search
  • Three step search (TSS) 5, employs rectangular
    search patterns with different sizes.
  • Koga et al 5 introduced this algorithm in 1981.
  • It became very popular because of its simplicity
    and also robust and near optimal performance.
  • It searches for the best motion vectors in a
    coarse to fine search pattern.
  • One problem that occurs with the three step
    search is that it uses a uniformly allocated
    checking point pattern in the first step, which
    becomes inefficient for small motion estimation.

18
Comparative study of Motion Estimation (ME)
Algorithms
  • Experimental results 5 show that the block
    motion field of a real world image sequence is
    usually gentle, smooth and varies slowly. This
    results in center-biased global minimum
    distribution instead of uniform distribution.
  • So, the checking point pattern should also be
    center- biased.

19
Comparative study of Motion Estimation (ME)
Algorithms
  • New three step search (NTSS)
  • To remedy the problem with TSS, NTSS was
    proposed.
  • It employs a center-biased checking point
    pattern in the first step, which is derived by
    making the search adaptive to the motion vector
    distribution, and a halfway-stop technique to
    reduce the computation cost.
  • Fig 7-a 5 and Fig 7-b 5 explain the
    implementation.
  • In Fig 7-a, filled circles are the checking
    points in the first step of TSS, squares are the
    8 extra points added in the first step of NTSS,
    and triangles explain how the second step search
    is performed if the minimum BDM (block distortion
    measure) in the first step is at one of the 8
    neighbors of the window center.
  • Fig 7-b shows the block diagram of NTSS
    algorithm.
  • Fig 7 Implementation details of NTSS Algorithm
    5

20
Comparative study of Motion Estimation (ME)
Algorithms
  • Four step search
  • This algorithm also exploits the center-biased
    property in its search.
  • The computational complexity of the four-step
    search is less than that of the new three step
    search, while the performance in terms of quality
    is as good.
  • It is also more robust than the three step search
    and it maintains its performance for image
    sequences with complex movements like camera
    zooming and fast motion. Hence it is a very
    attractive strategy for motion estimation.
  • Fig. 8 Search patterns of the 4SS (a) First
    step, (b) second/third step (c)second/third step
    (d) fourth step 13

21
Comparative study of Motion Estimation (ME)
Algorithms
  • Search points needed (considering the
    displacement parameter p in -7,7 range)-
  • Minimum 98 17
  • Worst case 9558 27 (Large motion)
  • Thus computational complexity is just two block
    matches more compared to TSS and lesser compared
    to 33 block matches in NTSS.
  • Performance comparable to NTSS.

Fig. 9
22
Comparative study of Motion Estimation (ME)
Algorithms
  • 2-D Logarithmic search
  • Although 2-D logarithmic algorithms require more
    steps than the three step search, it can be more
    accurate, especially when the search window is
    large6.
  • This procedure requires only searching 13 to 21
    locations.
  • Displacement parameter p 5

So n 2 for p5
23
Comparative study of Motion Estimation (ME)
Algorithms
  • One-at-a-time search (OTS)
  • One-at-a-time search (OTS) is a basic
    two-variable optimization search method.
  • Each variable is adjusted with the other fixed.
  • Direction of search in each step will be parallel
    to one of the coordinate axes.
  • Fig. 11 11 illustrates the OTS for a convex
    function in two variables.
  • Fig 11 Illustration of one at a time search 11

24
Comparative study of Motion Estimation (ME)
Algorithms
  • Conjugate direction search (CDS)
  • An extension of the one-at-a-time search is the
    conjugate direction search (CDS) as shown in Fig
    12 11.
  • The directions of search, rather than being kept
    fixed, are changed to improve the search
    efficiency.
  • The CDS obtains n linearly independent conjugate
    direction vectors, n being the number of
    variables in the objective function.
  • A succession of one-at-a-time searches in each of
    n sets of independent directions starting with
    the coordinate directions is carried out.

25
Comparative study of Motion Estimation (ME)
Algorithms
  • Diamond search (DS)
  • As indicated in Table 1, about 52.76 to 96.09
    of the motion vectors are enclosed in a circular
    support with a radium of 2 pels and centered on
    the zero-motion position.
  • TSS uses a sparse searching point pattern. This
    could yield the search path in wrong direction
    and thus give a wrong optimum point.
  • Other methods which use a smaller search pattern
    can cause the MDB to be trapped to a local
    minimum.

Table 1 Motion vector distribution aggregately
measured at various motion distances (in pel)
with regard to the center position using the Full
Search algorithm based on MSE matching criterion.
15
26
Comparative study of Motion Estimation (ME)
Algorithms
  • Diamond search (DS) (contd..)
  • Fig 14 An appropriate search pattern
    supportcircular area with a radium of 2 pels.
    The 13 crosses show all possible checking points
    within the circle 15

27
Comparative study of Motion Estimation (ME)
Algorithms
  • The DS algorithm employs two search patterns as
    illustrated in Fig. 15, which are derived from
    the crosses () in Fig. 14.
  • The first pattern, called large diamond search
    pattern (LDSP), comprises nine checking points
    from which eight points surround the center one
    to compose a diamond shape.
  • The second pattern consisting of five checking
    points forms a smaller diamond shape, called
    small diamond search pattern (SDSP).
  • In the searching procedure of the DS algorithm,
    LDSP is repeatedly used until the step in which
    the minimum block distortion (MBD) occurs at the
    center point.
  • The search pattern is then switched from LDSP to
    SDSP as reaching to the final search stage.
  • Among the five checking points in SDSP, the
    position yielding the MBD provides the motion
    vector of the best matching block.

28
Comparative study of Motion Estimation (ME)
Algorithms
Fig. 17 Diamond search path example which leads
to the motion vector (-4, -2) in five search
steps. There are 24 search points in total -
taking 9, 5, 3, 3 and 4 search points in each
step sequentially. 15
29
Comparative study of Motion Estimation (ME)
Algorithms
  • Feature-matching
  • Feature-matching algorithms can be seen as
    matching in meta information extracted from the
    current block and search area pels as shown in
    Fig.18 10
  • In most of the feature matching algorithms, the
    meta information has a higher entropy than the
    original information, thus reducing the
    arithmetic computational load, as well as the
    memory bandwidth within the matching process.
  • The feature extraction is performed by
    morphological filters, reduced pel resolution or
    projection methods.
  • Integral projection matching (IPM), projection
    three step search (PTSS), successive elimination
    algorithm (SEA), binary block matching (BBM), bit
    plane matching (BPM), block feature matching
    (BFM) are some of the algorithms based on feature
    matching.

30
Comparative study of Motion Estimation (ME)
Algorithms
  • 3. Fixed size block motion estimation (FSBME) and
    variable size block motion estimation (VSBME) 3
  • The advantage of the former is that there is no
    need to provide segmentation information. However
    it is not very efficient when we have images that
    have both really fast and slow motions.
  • In this case the latter method is more useful
    because we can segment the image into smaller
    blocks in areas where there is complex motion and
    larger blocks when there is very little motion.

31
Comparative study of Motion Estimation (ME)
Algorithms
  • 3. Simulation
  • The simulation results for exhaustive search,
    TSS, NTSS, 4SS, DS on different test sequences
    have been tabulated.
  • Cronkite (256X256), toy vehicle (512X512),
    chemical plant (256X256) and caltrain test
    sequences 19 were used in the simulation. These
    are obtained from the digitization of 24 frames
    per second motion pictures.
  • The source code for the various algorithms is
    available from Matlab central file exchange 20.
    This code was modified for different images.
  • The distance criterion method used is MAE.
  • Block size of 8X8 (16X16) is used. Search window
    size is (p p block size) X (p p block
    size) where p (7) is the displacement parameter.
  • The various algorithms are compared based on
    their computational complexity and PSNR.

32
Comparative study of Motion Estimation (ME)
Algorithms
Table 2 Average number of computations needed
for different ME algorithms
33
Comparative study of Motion Estimation (ME)
Algorithms
Table 3 Average PSNR (dB) for different ME
algorithms
34
Comparative study of Motion Estimation (ME)
Algorithms
  • Simulation results show that ES gives the best
    performance (highest PSNR) at the cost of high
    computational complexity. Motion estimation
    constitutes about 60 of the total computational
    load in encoders, so exhaustive search algorithm
    is not the most feasible technique to employ
    motion estimation.
  • TSS algorithm gives good performance but it can
    not always detect small motions.
  • NTSS is good for slow motions but its
    computational complexity is higher than that of
    TSS for large motion.
  • 4SS has a small initial step and is good to
    detect small motion. Its computational complexity
    is comparable to that of TSS and lower than that
    of NTSS.
  • DS performs well both computationally and in
    terms of quality.

35
Comparative study of Motion Estimation (ME)
Algorithms
  • References
  • 1 Z. Wei et al Efficient fast motion
    estimation algorithm for H.264 based on
    polynomial model, IMACS Multiconference on
    computational engineering in System Applications,
    pp. 1654-1657, Oct. 4-6, 2006. 
  • 2 T. Huang, Y. Hsu and R. Tsai Interframe
    coding with general two-dimensional motion
    compensation, IEEE International Conference on
    ICASSP 82 (Acoustics, Speech and Signal
    Processing), vol. 7, pp. 464 - 466, May 1982
  •  
  • 3 M. Manikandan, P. Vijayakumar and N.
    Ramadass, Motion estimation method for video
    compression - an overview, Wireless and Optical
    Communications Networks, IFIP Intl Conference,
    pp. 5-11, 13 April 2006. 
  • 4 H. Ferhatosmanoglu et al, Approximate
    nearest neighbor searching in multimedia
    databases, Data Engineering, 2001. Proceedings.
    17th International Conference on, pp. 503 511,
    2-6 April 2001. 
  • 5 R. Li, B. Zeng, and M. L. Liou, A new
    three-step search algorithm for block motion
    estimation IEEE Transactions on Circuits and
    Systems Video Technology,vol. 4, pp. 438443,
    Aug. 1994. 
  • 6 J. R. Jain and A. K. Jain, Displacement
    measurement and its application in interframe
    image coding IEEE Transactions on
    Communications, vol. 29, pp. 1799-1808, Dec.
    1981.

36
Comparative study of Motion Estimation (ME)
Algorithms
  • 7 J. Kim and S. Yang, An efficient search
    algorithm for block motion estimation IEEE
    Workshop on Signal Processing Systems, pp.
    100-109, Oct. 1999.
  •  
  • 8 C. Zhu et al., Enhanced hexagonal search for
    fast block motion estimation, IEEE Transactions
    on Circuits and Systems for Video Technology,
    vol. 14, pp. 1210 - 1214, Oct. 2004.
  •  
  • 9 E. Fernandez et al., Reducing motion
    estimation complexity in MPEG-2 to H.264
    transcoding, IEEE International Conference on
    Multimedia and Expo, 2007 ,pp. 440-443, 2-5 July
    2007.
  •  
  • 10 K. Peter, Algorithms,complexity analysis
    and vlsi architectures for MPEG-4 motion
    estimation, Kluwer Academic Publishers, Boston,
    1999.
  •  
  • 11 R. Srinivasan and K. R. Rao, Predictive
    coding based on efficient motion estimation IEEE
    Transactions on Communications , vol. 33, pp.
    888-896, Aug. 1985.
  • 12 I. Amre et al, An efficient variable block
    size selection scheme for the H.264 motion
    estimation 6th International Workshop on
    System-On-Chip for Real-Time application, pp.
    5-9, Dec. 2006.

37
Comparative study of Motion Estimation (ME)
Algorithms
  • 13 P. Lai-Man and M. Wing-chung, A novel
    four-step search algorithm for fast
    blockmatching, IEEE Transactions On Circuits and
    Systems for Video Technology, vol. 6, pp.
    313-317, Jun. 1996.
  •  
  • 14 H. Zhongli and M.L. Lieo, A high
    performance fast search algorithm for block
    matching motion estimation, IEEE Transactions on
    Circuits and Systems for Video Technology, vol.
    7, pp. 826-828, Oct. 1997.
  • 15 S. Zhu and K. K. Ma A new diamond search
    algorithm for fast block-matching motion
    estimation, IEEE Transactions on Image
    Processing, vol. 9, pp. 287-290, Feb. 2000.
  • 16 Y. Chan and W. Siu , An efficient search
    strategy for block motion estimation using image
    features, IEEE Transactions on Image Processing,
    vol. 10, pp. 1223-1238, Aug. 2001.
  • 17 J. Lee et al, An efficient architecture for
    motion estimation and compensation in the
    transform domain, IEEE Transactions on Circuits
    and Systems for Video Technology, vol. 16, pp.
    191-201,Feb. 2006.
  • 18 C. Cheung and L. Po, A hierarchical block
    motion estimation algorithm using partial
    distortion measure, Proceedings International
    Conference on Image Processing, vol. 3, pp.
    606-609, Oct. 1997
  • 19 Test sequences obtained from
  • http//sipi.usc.edu/database/database.cgi
    ?volumesequences
  •  
  • 20 Motion estimation algorithm source code
    available at Matlab central file exchange
  • www.mathworks.com/matlabcentral/fileexcha
    nge
  • 21 Phase correlation motion estimation
    report available at http//visilab.unime.it/iann
    i/slides_CV/liang_report-phase-correlation-motion-
    prediction.pdf
  •  

38
Comparative study of Motion Estimation (ME)
Algorithms
  • APPENDIX A
  • ACRONYMS
  • 2DLOG 2-d Logarithmic Search
  • BBM Binary Block Matching
  • BDM Block Distortion Measure
  • BFM Block Feature Matching
  • BPM Bit Plane Matching
  • DCT Discrete Cosine Transform
  • DS Diamond Search
  • FS Full Search
  • FSBME Fixed Size Block Motion Estimation
  • H.264/AVC Video Coding Standard
  • HPDS Hierarchical Partial Distortion Search
  • IPM Integral Projection Matching
  • MAE Mean Absolute Error
  • MB Macro Block
  • ME Motion Estimation
  • MSE Mean Square Error
  • MV Motion Vector

39
Comparative study of Motion Estimation (ME)
Algorithms
  • Questions?
About PowerShow.com