Topic 5' Human Faces

1
Topic 5. Human Faces

• Human face is extensively studied in vision.
  Depending on the applications, there are a
• long list of tasks 5
• Detection and Recognition
• Face detection (finding all faces
  in a picture), facial feature detection (eyes,
  lips, ),
• Face localization (detecting a
  single face in image),
• Face recognition or
  identification (from a database, classification)
• Face authentication (verifying
  claim, bank id), Age/gender recognition,
• Face tracking (location and pose
  over time)
• Facical expression recognition
  (affective states), aesthetic study.
• Modeling and Photorealistic Synthesis
• Appearance models, deformable
  templates, lighting models, facial action
  units,
• face hallucination (high
  resolution from low resolution),
• pose adjustment, image editing
  (removing wrinkles, eye glass, red-eye etc.)
• 3. Artistic rendering
• Sketch, portrait, caricature,
  cartoon, painting,

2
Face Image Databases
The CMU Rowley dataset
3
Face Image Databases
The CMU Schneidrman and Kanade Dataset
4
References.
1. P. Hallinan, G. Gordon, A. Yuille, P.
Giblin, and D. Mumford, 2D and 3D Patterns
of the Face, A.K. Peters, Ltd. Book
chapters 2-4. (handouts). 2. D.H. Ballard,
"Generaling the Hough transform to detect
arbitrary shapes", (in handbook). 3. P. Viola
and M. Jones, "Robust Real Time Object
Detection", 4. F. Fleuret and D. Geman, "
Coarse-to-fine face detection", IJCV
41(1/2),2001. 5. M.H. Yang, D. Kriegman, N.
Ahuja, Detecting faces in images, a survey,
PAMI vol.24,no.1, January, 2002. 6
T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active
Appearance Models", ECCV 1998 7. C. Liu, S. C.
Zhu, and H. Y. Shum, "Learning inhomogeneous
Gibbs models of faces by minimax entropy", ICCV
2001. 8. Y. Tian, T. Kanade, and J. Cohn,
"Recognizing action units for facial expression
analysis" PAMI, Feb, 2001. 9. H.
Chen, Y. Q. Xu, H. Y. Shum, S. C. Zhu, and N. N.
Zhen, "Example-based facial sketch generation
with non-parametric sampling", ICCV 2001.
5
Outline

• We proceed in three steps
• A survey on face detection and recognition
  techniques
• Mathematical models of face images
• 3. Face synthesis photorealistic and
  non-photorealistic.

6
Face Detection Methods 5
7
Face vs non-face Clsutering
6 clusters in a 19 x19 space (Sung and Poggio)
8
Distance Measure
D2
D1
For each input image, it measures two distances
for each cluster center D1 is the
Mahalanobis distance and D2 is the Euclidean
distance. Thus Sung and poggio have 2 x 6 x 2
24 features for classification in a multiple
layer perceptron.
9
Deformable Face Template
Deformable face template by Fishler and
Elschlager 1973. M. Fishler and R. Elschlager,
The representation and matching of pictorial
structures, IEEE Trans. on Computer.
Vol.C-22, 67-92, 1973.
10
Local Deformation and Global Transform
Geometric variations of faces (Hallinan, Yuille,
Mumford et al)
11
Deformable Model of Facial Features
Eye template using parabolic curves by Yuille et
al 1989-92. A.L.Yuille, D. Cohen, and
P.Hallinan, Feature extraction from faces using
deformable templates, CVPR 89, IJCV 92.
We can derive meaningful diffusion equations from
the energy functionals.
12
Upper Face Action Units
13
Lower Face Action Units
14
Templates for Various States
15
Templates for Various States
16
Features for Action Unit Recognition
17
Classification from Feature Vector
18
Recognition Rate
19
Apparence Model Landmarks on a face
400 images each labeled with 122 points.
20
Eigen-vectors for Geometry and Photometry
21
Apparence Model
22
Face Localization and Recognition
23
A Linear HMM Model for Face
24
Face Detection
25
Sample of the 4D space
26
Multi-scale Detection
27
Edge Features
28
Decision Tree
29
Examples of Decision Trees
30
Bounds Analysis
31
Some Examples
32
Face Prior Learning Experimental Details

• 83 key points defined on face
• 720 individuals with all kinds of types
• Dimension reduced to 33 by PCA
• 40000 samples drawn by the inhomogeneous Gibbs
  sampler in each Monte Carlo integration
• 50 features pursuit
• Total runtime about 5 days on a PIII 667, 256MB
  PC

33
Obs Syn Samples (1)
Observed faces
Synthesized faces without any features
34
Synthesis Samples
Synthesized faces with 10 features
Synthesized faces with 20 features
35
Synthesis Samples
Synthesized faces with 30 features
Synthesized faces with 50 features
36
50 Observed Histograms
37
50 Synthesized Histograms