Active Appearance Models

1
Active Appearance Models

• Dhruv Batra
• ECE CMU

2
Active Appearance Models

1. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active
   Appearance Models", in Proc. European Conference
   on Computer Vision 1998 (H.Burkhardt B. Neumann
   Ed.s). Vol. 2, pp. 484-498, Springer, 1998
2. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active
   Appearance Models", IEEE PAMI, Vol.23, No.6,
   pp.681-685, 2001
3. G.J. Edwards, A. Lanitis, C.J. Taylor, T. F.
   Cootes. Statistical Models of Face Images
   Improving Specificity, BMVC (1996)

3
Essence of the Idea

• Interpretation through synthesis
• Form a model of the object/image (Learnt from the
  training dataset)

I. Matthews and S. Baker, "Active Appearance
Models Revisited," International Journal of
Computer Vision, Vol. 60, No. 2, November, 2004,
pp. 135 - 164.
4
Essence of the Idea (cont.)

• Explain a new example in terms of the model
  parameters

5
So whats a model
Model
texture
Shape
6
Active Shape Models
training set
7
Texture Models
warp to mean shape
8
Random Aside

• Shape Vector provides alignment

43
Alexei (Alyosha) Efros, 15-463 (15-862)
Computational Photography, http//graphics.cs.cmu.
edu/courses/15-463/2005_fall/www/Lectures/faces.pp
t
9
Random Aside

• Alignment is the key

1. Warp to mean shape 2. Average pixels
Alexei (Alyosha) Efros, 15-463 (15-862)
Computational Photography, http//graphics.cs.cmu.
edu/courses/15-463/2005_fall/www/Lectures/faces.pp
t
10
Random Aside

• Enhancing Gender

more same original androgynous more opposite
D. Rowland, D. Perrett. Manipulating Facial
Appearance through Shape and Color, IEEE
Computer Graphics and Applications, Vol. 15, No.
5 September 1995, pp. 70-76
11
Random Aside (cant escape structure!)
Antonio Torralba Aude Oliva (2002) Averages
Hundreds of images containing a person are
averaged to reveal regularities in the intensity
patterns across all the images.
Alexei (Alyosha) Efros, 15-463 (15-862)
Computational Photography, http//graphics.cs.cmu.
edu/courses/15-463/2005_fall/www/Lectures/faces.pp
t
12
Random Aside (cant escape structure!)
Tomasz Malisiewicz, http//www.cs.cmu.edu/tmalisi
e/pascal/trainval_mean_large.png
13
Random Aside (cant escape structure!)
100 Special Moments by Jason Salavon
Jason Salavon, http//salavon.com/PlayboyDecades/P
layboyDecades.shtml
14
Random Aside (cant escape structure!)
Every Playboy Centerfold, The Decades
(normalized) by Jason Salavon
1960s
1970s
1980s
Jason Salavon, http//salavon.com/PlayboyDecades/P
layboyDecades.shtml
15
Back (sadly) to Texture Models
raster scan
Normalizations
16
PCA Galore
Reduce Dimensions of shape vector
Reduce Dimension of texture vector
They are still correlated repeat..
17
Object/Image to Parameters
modeling
80
18
Playing with the Parameters
First two modes of shape variation
First two modes of gray-level variation
First four modes of appearance variation
19
Active Appearance Model Search

• Given Full training model set, new image to be
  interpreted, reasonable starting approximation
• Goal Find model with least approximation error
• High Dimensional Search Curse of the dimensions
  strikes again

20
Active Appearance Model Search

• Trick Each optimization is a similar problem,
  can be learnt
• Assumption Linearity
• Perturb model parameters with known amount
• Generate perturbed image and sample error
• Learn multivariate regression for many such
  perterbuations

21
Active Appearance Model Search

• Algorithm
• current estimate of model parameters
• normalized image sample at current estimate

22
Active Appearance Model Search

• Slightly different modeling
• Error term
• Taylor expansion (with linear assumption)
• Min (RMS sense) error
• Systematically perturb and estimate by
  numerical differentiation

23
Active Appearance Model Search (Results)
24
(No Transcript)