Automatic Face Feature Localization for Face Recognition

1
Automatic Face Feature Localization for Face
Recognition

• Christopher I. Fallin
• honors thesis defense May 1, 2009
• advisor Dr. Patrick J. Flynn

2
Outline

• Face recognition
• Methods of evaluation
• Elastic Bunch Graph Matching
• Gabor Jets
• Bunch Graphs and Feature Localization
• My Contributions Automatic Fiducial Points
• Information Content model
• Fiducial point placement
• results

3
Face Recognition

• Subfield of biometrics
• life (bio)
• measure (metric)
• Extract identifying information from measures of
  human traits
• Face recognition digital images of face
• 2D, 3D, infrared, multimodal,

http//www-users.cs.york.ac.uk/nep/research/3Dfac
e/tomh/3DFaces.jpg
4
Face Recognition Evaluation
Image Set
5
ROC curves
System Decision
Y N
Actual Y N
http//en.wikipedia.org/wiki/FileRoc-general.png
used under terms of GNU FDL
6
EER 11.1
7
Rank-one Score
Gallery
  A B C D E F G
A 0.89 0.70 0.10 0.52 0.34 0.48 0.37
B 0.70 0.73 0.45 0.82 0.12 0.43 0.44
Probe C 0.10 0.45 0.92 0.89 0.23 0.82 0.13
D 0.52 0.82 0.89 0.56 0.20 0.38 0.14
E 0.34 0.12 0.23 0.20 0.82 0.52 0.23
F 0.48 0.43 0.82 0.38 0.52 0.84 0.11
G 0.37 0.44 0.13 0.14 0.23 0.11 0.99

Rank-one Rank-one 5/7 71.4 5/7 71.4
8
Elastic Bunch Graph Matching (EBGM)

• Wiskott et al., USC/Bochum, mid-90s
• Basis of ZN-Face, successful commercial system
• We use Face Identity Evaluation System, from
  Colorado State
• Face features represented by Gabor filter
  responses
• Features are localized
• Fit an elastic graph onto the features by
  localization local optimization problems

9
A Face Graph
Wiskott99
10
Gabor Jets

• Vector of filter responses to 40 Gabor kernels
• 5 wavelengths
• 8 orientations
• Each is complex-valued
• Gabor jets capture information well Gokberk et
  al. get 91 rank-one with fixed grid
• On FERET 78.5 max, with 12 grid points

11
Bunch Graphs

• Each feature has a bunch of canonical jets
• Represents typical features
• Best-match at each feature point for novel images

Wiskott99
12
Feature Localization

• Initial alignment eye locations known a-priori
• Overlay bunch graph with average edge lengths
• Take Gabor jets pick best match in each bunch
• Localize based on displacement estimation (local
  optimization problem)

13
The Idea Automatic Fiducial Point Placement

• Bunch graph training requires manual fiducial
  point placement
• 70 images, 25 points
• Why not statistically determine optimal features
  to match on?
• We can align/normalize all faces and take some
  statistical measure at each point in face space
  to determine goodness
• Replaces training step back-end algorithm is
  identical

14
Related Work

• Gokberk et al. choosing fiducial points with
  genetic algorithms
• But their chosen points are global
• Same goal as our system, excluding
  prelocalization
• Salient Points
• Wavelet-based approach to image retrieval
• Choras et al., 2006 similar approach with
  goodness function, but no EBGM

15
Information Content Variance Model

• Compute goodness function over face-space
• Inter-subject variance over intra-subject
  variance
• Self-normalizing unitless measure
• Requires multiple images per subject

16
Computing the goodness function

• FRGC 5404 images 700 MB, 128x128 grayscale (7
  GB before normalization)
• Each pixel 12 seconds, on fast Athlon 64
• Split into 128 Condor jobs
• Each pixel is independent easy
• Pre-normalize image set, dump to fast-loading
  binary format (single file)
• Run Condor jobs three hours
• Post-processing to reassemble results

17
Fiducial Point Placement

• Random placement with probability density
• Compute gradient of goodness function
• Probability is product of gradient and goodness
• Place points sequentially, decay probability
  around points exponentially
• Mirror-point constraint mirror placements across
  centerline, or snap to center

18
Prelocalization Pseudo-Bunches

• Displacement estimation requires canonical
  feature jet from bunch
• We cant provide this if we have no knowledge of
  feature
• Solution fake a jet bunch
• Make educated guess with K-means clustering on
  jets from all images at given point
• Then, run displacement estimation to prelocalize
  points on each image

19
Results
FERET

• Competitive with original, manual points
• In both cases, automatic training points yield
  only 1 performance drop
• With no human training!
• Prelocalization did not work as intended
• Success without this suggested by Gokberks
  results

FRGC
20
ROC curves

• FERET

• FRGC

(EER 11.4) (orig 11.1)
(EER 31.4) (orig 34.8)
21
Prelocalization causes for failure

• Poor pseudo-bunch clustering K-means often found
  optimal clustering at self-imposed cap of N/10
  clusters
• Likely because initial jets are too far off
• Naïve localization single-step
• Bolme thesis compares several optimization
  algorithms
• Average displacement of 2.628 pixels larger than
  2.021 pixels in manual points

22
Future Work

• More sophisticated prelocalization
• Look at pseudo-bunch statistics to determine
  failure mode in more detail
• Look at per-fiducial point statistics to
  determine where performance is weak
• Investigate are manual pts a theoretical limit,
  or can we exceed them?
• Try new image classes test claim of genericism

23
Questions?

• Email cfallin_at_c1f.net
• Full thesis and source code will be posted
  online http//c1f.net/research/mark5/

24
Distance Metrics on Jets

• Phase-insensitive magnitude only
• Selects best jet in bunch
• Phase-sensitive
• Can solve for displacement vector basis of
  localization
• Displacement estimation