Eigenfaces for Recognition By: Matthew Turk and Alex Pentland 1991 IEEE

1
Eigenfaces for RecognitionByMatthew Turk and
Alex Pentland1991 IEEE

• Presented byShane Brennan
• 5/02/2005

2
Defining Characteristics

• A holistic based method, meaning it uses the
  whole face region in the recognition system.
• Emphasizes significant features of the image,
  which may or may not coincide with features
  significant in human perception such as the eyes,
  nose, ears, or mouth.
• Performs well under a variety of lighting
  conditions.
• Performs poorly under variations in image scale.

3
Theory Behind Eigenfaces

• Each input image of size NxN (intensity values)
  can be viewed as a vector of dimension N2.
• Images of faces, being similar in configuration,
  will not be distributed randomly in this N2
  dimensional space.
• Use PCA to project input image into a
  lower-dimension subspace. The vectors obtained
  from PCA are what we call the eigenvectors, or
  eigenfaces.

4
Calculating Eigenfaces

• Take a collection of images, these are the people
  to be recognized. The images are also used as a
  sort of training data.
• Take the M training image vectors and average
  them to find ? 1/M ?n 1 to M Gn
  where Gn is the nth image vector.
• Each face differs from the average by Fi Gi - ?.

5
Calculating Eigenfaces, continued

• These M vectors (Fi) then undergo Principal
  Component Analysis to find a set of M orthonormal
  vectors, defined as un, and their associated
  eigenvalues, ?n.
• These are the eigenvectors of the Covariance
  Matrix C 1/M ?n 1 to M FnFnT AAT.
  Where A F1 ... FM.

6
Finding C

• C is an N2 x N2 matrix. To calculate this matrix
  is computationally expensive.
• If M ltlt N2 then there will only be M 1
  eigenvectors which have non-zero eigenvalues. So
  can solve an M x M matrix instead.Consider the
  eigenvectors, vi, of ATA such that
  ATAvi µivi which yields
  AATAvi µiAvi
• From this it can be seen that Avi are the
  eigenvectors of C AAT (and µi are the
  eigenvalues).

7
Finding C, continued

• Following this, form the M x M matrixL ATA
  and calculate the M eigenvectors of L, which are
  referred to as vi.
• To form the eigenfaces ui use the following
  equation ui ?k 1 to M vikFk for i 1 ...
  M.
• This takes the computation down from the order of
  N2 to the order of M.

8
Classifying An Image

• Given the set of M eigenfaces, choose the M '
  eigenfaces that have the highest associated
  eigenvalues.
• M ' can be a small number (on the order or 10
  50).
• Take a new face image, G, and project it into
  face space by the operationWk ukT (G - ?)
  for k 1 to M '.
• The weights (Wk) form a vector OT W1 ... Wk
  which describes the contribution of each
  eigenface in representing the input face image.

9
Classifying An Image, continued

• Take OT and determine which face class, if any,
  best describes the input face.
• To do this, find the Euclidian distance ek O
  - Ok where Ok is the weight vector describing
  the kth training face image. A face is identified
  as person k if ek is below some certain threshold
  Te.
• If each ek is greater than Te then the input
  image is determined to not be any known face.
• In addition, if the input vector lies far from
  face space, it can be classified as not even
  being a face image.

10
The average face (left), and several eigenfaces
(right).
11
An input face image and its projection onto face
space.
12
Detecting Faces

• Given a large input image of size N x N the
  locations of faces can be found.
• For each S x S subregion of the image, project F
  onto the facespace with the operationFf ?k
  1 to M ' Wkuk. The distance of the local
  subregion from facespace is e(x,y) F
  Ff.
• Regions of the image with a low e (below a given
  threshold) most likely contain a face (which is
  centered in the S x S subregion).
• However, this calculation is extremely expensive
  (on the order of N2). A more efficient method is
  needed.

13
Detecting Faces, continued

• e2 F Ff2 (F Ff)T(F Ff) FTF
  FTFf FfT(F Ff) FTF FfTFf Since Ff is
  perpendicular to (F Ff).
• Since Ff is a linear combination of the
  eigenfaces, and the eigenfaces are orthonormal
  vectors we can caluclate FfTFf as FfTFf ?i
  1 to L Wi2
• So e2(x,y) FTF ?i 1 to M ' Wi2

14
Detecting Faces, continued

• ?i 1 to M ' Wi2 ?i 1 to M ' FTui ?i 1
  to M ' G ?T ui ?i 1 to M ' GTui
  ?Tui ?i 1 to M ' I(x,y) x ui
  ?TuiWhere I(x,y) is the overall (large) input
  image.I(x,y) x ui represents the correlation
  between I(x,y) and ui.
• FTF G ?T G ? GTG 2?TG ?T? GTG
  2G x ? ?T?Where G x ? is the correlation
  between G and ?.

15
Detecting Faces, continued

• Bringing this all togethere2(x,y) GTG - 2G x
  ? ?T? ?i 1 to M ' I(x,y) x ui
  ?T x ui
• ? and ui are fixed, so ?T? and ?T x ui can be
  computed ahead of time.
• This means that only M '1 correlations must be
  computed, as well as the GTG term. (Note that M '
  is typically on the order of 10 - 50)
• GTG is computed by squaring I(x,y) and then
  summing the squared values in the local subregion.

16
The input image (left), and the corresponding
face map (right). Dark areas indicate the
presence of a face.
17
Additional Features

• One can improve recognition by taking training
  images at different sizes, and at different
  angles.
• This creates a number of different face spaces.
• The face can then be recognized under varying
  conditions of size and rotation.
• This also allows the algorithm to identify the
  orientation of the face.
• Use a Gaussian window to remove the background to
  ensure that features of the background are not
  significant in the eigenface.

18
Experiments and Results

• In a study using 16 subjects under a variety of
  lighting, size, and head orientation and Te set
  to infinity (no input images rejected as unknown)
  the following results were obtained96 correct
  classification over lighting variation85
  correct classification over orientation
  variation64 correct classification over size
  variation
• Setting Te for 100 accurate recognition results
  in some images of known individuals being
  rejected as unknown, these rates are19 under
  variation of lighting39 under variation of
  orientation60 under variation of size

19
Thank You!