View-based Object Modeling, Registration and Verification

1
View-based Object Modeling, Registration and
Verification

• Lixin Fan
• School of Computing
• National University of Singapore

2
Objective

• Object registration For given object images,
  mark out object features (e.g. eyes, mouths and
  noses).

3
Problem Formulation

• Model-based image matching

4
Presentation Outline

• Objective
• Problem Formulation
• Object Modeling
• Similarity Measure
• Image Matching Algorithm
• Applications
• Contribution
• Acknowledgement

5
View-based Object Modeling

6
Image variations

• Textural variations
• Differences in intensity values between
  corresponding pixels.
• Structural variations
• Spatial displacements between corresponding
  feature points, encoded by feature point
  correspondence map (FPCM)

7
Textural modeling

• Eigenface model Turk and Pentland 91

8
Structural (shape) modeling

• Feature Point Correspondence map model

9
Combine textural and structural variations

• Feature-based image warping (Bookstein 89,
  Beier and Neely 92,)

10
Non-linearity of Model
Figure 5 Distribution of the first 3 principle
components of varying pose faces (pose 1 right
rotation pose 2 minor right rotation post 3
frontal post 4 minor left rotation and pose 5
left rotation).
11
Object Models for Registration

• 2D Object Models (view-based)
• Active Appearance Model (Cootes and Taylor 98/99,
  Wolfson Image Analysis Unit, University of
  Manchester).
• Face Vectorization (Beymer 95, Jones 96, Vetter
  and Poggio 97, MIT AI Lab).
• 3D Object Models
• Alignment by MMI (Paul Viola 95, MIT AI Lab)
• Ellipsoidal Head Model (Essa and Pentland 97,
  MIT, Media Lab)
• Anthropometric Face model (DeCarlo and Metaxas
  96, Univ. of Philadelphia).

12
Presentation Outline

• Objective
• Problem Formulation
• Object Modeling
• Similarity Measure
• Image Matching Algorithm
• Applications
• Contribution
• Acknowledgement

13
Similarity (Error) Measure

• How to compare images

compare ?
model image

• Intensity (texture) based measures Sum of
  squared distance (SSD), normalized correlation,
  etc.
• Structural (feature) based measures Hausdorff
  distance and its variations.

14
Structurally normalized textural difference

• Structurally normalized textural difference
• Difficulties Intensity differences
  structural differences.

15
Combined Feature-Texture Similarity Measure (FTSM)

• Combined structural difference and structurally
  normalized textural difference (Fan and Sung
  2000)
• Closest edge point match to create feature points
  correspondence map C

?
I1
I2
C
16
Structure difference (cont.)

• The average length of displacement vectors
  encodes structural difference (Huttenlocher 93)
• Implementation issues
• Distance transformation to speed up point
  matching (Borgefors 86).
• Dynamic point matching algorithm to avoid
  one-many point matching (paper in preparation).

17
Object verification

• FTSMs can be used to verify the existence of
  objects (Fan and Sung 2000)

18
Presentation Outline

• Objective
• Problem Formulation
• Object Modeling
• Similarity Measure
• Image Matching Algorithm
• Applications
• Contribution
• Acknowledgement

19
Image Matching Algorithm

• Initialize
• Fix , estimate
• Fix , estimate
• Iterate 2 and 3, until reaches
  minima.

20
FPCM based hill-climbing

1. Determine a search direction by projecting C
   into the eigenshape space
2. Set new
3. Iterate 1 and 2, until
   reaches minima.

21
FPCM based hill-climbing

• FPCM based hill-climbing
• better avoids local minima
• converges quickly.

22
Presentation Outline

• Objective
• Problem Formulation
• Object Modeling
• Similarity Measure
• Image Matching Algorithm
• Applications
• Contribution
• Acknowledgement

23
Application(1) Varying pose face alignment

• Example images
• Training images of MIT Beymer face database
• 60 different people under 5 poses (300 images),
  ranging between -600, 600 left-right rotations
  and -100, 100 down-up rotation
• Size 100 x 100 pixels
• Varying pose face model
• Number of eigenfaces N20
• Number of eigenshapes M10

24
Application(1) Varying pose face alignment

• Test images
• 50 testing images of Beymer database
• 20 face images (under varying lighting
  conditions)
• Alignment results

Face Good Fair Misaligned
70 50 (71.4) 19 (27.1) 1 (1.4)
Good at most 2 feature points, out of 24 points,
are misaligned (i.e. 10 pixels away, or 1/10th of
image size) Fair at most 5 feature points are
misaligned Misaligned otherwise.
25
Application(1) Varying pose face alignment
26
Application(1) Varying pose face alignment

• The matching converges in few iterations even
  when initial estimate is far off.

9
Iterations 0
1
5
27
Application(2) Face Detection and Verification
28
Application(2) Face Detection and Verification
29
Application(2) Face Detection and Verification
False detection elimination
30
Application(2) Face Detection and Verification
31
Application(2) Face Detection and Verification
32
Application(2) Face Detection and Verification
Frame Face size Good Fair Mis.
Seq. 1 100 100x100 54 42 4
Seq. 2 100 100x100 48 47 5
Seq. 3 100 80x80 65 35 0
Seq. 4 100 50x50 54 45 1
Seq. 5 100 80x80 49 46 5
Seq. 6 100 80x80 60 37 3
Overall 600 - 330 252 18
33
Application(3) Pedestrian Contour Registration

• Example images
• Training images of MIT Pedestrian database
• 207 different people, with various body shapes
  and unconstrained backgrounds
• Size 64 x 128 pixels.

34
Application(3) Pedestrian Contour Registration

• Pedestrian image model
• Number of eigenvector of shape-normalized
  pedestrian images N20

35
Application(3) Pedestrian Contour Registration

• Number of eigenshapes M15

36
Application(3) Pedestrian Contour Registration -
Preliminary results
37
Contribution

• We proposed to deal with the object registration
  problem within a general image-matching
  framework. Experimental results show its
  effectiveness.
• The view-based model is capable of capturing both
  textural and structural image variations.
• A combined Feature-Texture similarity measure can
  deal with large amount of shape variations, even
  when initial shapes are far off.
• Feature Point Correspondence Map based
  hill-climbing is fast, and better avoid local
  minima.

38
Future Work

• More reliable feature extraction, possibly
  perceptual grouping.
• More reliable correspondence establishment.
• Application to more objects.

39
Acknowledgement

• Assoc. Prof. Sung Kah Kay
• Dr. Ng Teck Khim
• Friends at Soc
• Annie, Huizhong, Li Rui, Rini, Luping, Indriyati,
  Handoko, Tang mengting, Manoranjan Dash, Terrence
  Tan, Jack Yeo, Nadeem, Jian wei, Wang bing, Zhao
  Yunlong, and many more
• My family.