A Literature Review of Imagebased Face Recognition

1
A Literature Review of Image-based Face
Recognition
Quan Ju PhD student Department of Computer
Science The University of York
2
Background Introduction

• What is Face Recognition?
• Face Recognition is a popular application of
  computer vision in recent years.
• Not only the computer science researchers, but
  also the psychologists and neuroscientists are
  involved in this area.
• Why do we need Face Recognition?
• Strong need for personal identification and
  recognition without the cooperation of the
  participants.
• Commercial, security and law applications require
  the use of face recognition technology.

3
Face Recognition Scenarios

• Face Verification
• Is this person who he says he is? One to one
  matching process.
• Face Identification
• Who is this person? One to many matching process
• Watch list
• Who are you looking for?

4
Difficulties in Face Recognition

• Head pose
• Illumination
• Facial expression
• Hair
• Aging problem
• Occlusion e.g. glasses, scarf etc.

5
Image-based Face recognition approaches

• Appearance-based face recognition
• Linear Analysis PCA,ICA,LDA
• Non-linear analysis
• Model-based face recognition
• Elastic Bunch Graph Matching
• 2D Morphable Model
• 3D Morphable Model

6
Linear Analysis

• Classical linear appearance-based analysis - PCA,
  ICA and LDA each has its own basis vectors of a
  high dimensional face image space.
• By using those linear analysis method, the face
  vectors can be projected to the basis vectors.
• Through the projecting from a higher dimensional
  input image space to a lower dimensional space,
  dimensionality of original input image space is
  reduced.
• The matching score between the test face image
  and training images can be achieved by
  calculation the differences between their
  projection vectors. The higher the score, the
  more similar between these two face images.

7
Image Space

• Image vector and image subspace
• x1 represents a pq image x is a matrix of
  image vectors.

Above is three 1x2 pixel image examples. Similar
images locate close together, otherwise they are
away from each other.
8
Principal Component Analysis

• The main idea of the principal component analysis
  is to find the vectors which best describe the
  distribution of face images within the entire
  image space.
• PCA is an orthogonal transformation of the
  coordinate system in which the pixels are
  described. The new coordinate values are
  principal component
• Face space is comprised of eigenfaces, which are
  the eigenvectors of the set of the faces.
• PCA is performed by projecting a new image into
  the subspace called face space spanned by the
  eigenfaces and then classifying the face by
  comparing its position in face space with the
  positions of known individuals.
• PCA aims to extract a subspace where the variance
  is maximized

9
Independent Component Analysis

• PCA derives only the most expressive features
  which are unrelated to actual face recognition,
  and in order to improve performance additional
  discriminant analysis is needed.
• ICA provide a more powerful data representation
  than PCA as its aim is to provide an independent
  rather than uncorrelated image decomposition and
  representation.
• ICA is a generalization of PCA.

10
Linear Discriminant Analysis

• Similar images projections are close together,
  different images projections locate far away when
  using PCA, but the projection from different
  classes of images are mixed together.
• LDA is also called Fisher Discriminant Analysis
• LDA is able to maximize the ratio of
  between-class distribution to that of
  within-class distribution.

11
Nonlinear analysis

• Linear discriminant methods are insensitive to
  the relationship among multiple pixels in the
  images. Some nonlinear relations may exist in a
  face image, especially under a complicated
  variation in viewpoint, illumination and facial
  expression which is highly nonlinear.
• To extract nonlinear features of images, Linear
  analysis method was extended to nonlinear
  analysis such as Kernel PCA, Kernel ICA and
  Kernel FLD etc.
• By using nonlinear analysis approaches the
  original input image space is projected
  nonlinearly onto a high dimensional feature
  space. In this high dimensional space, the
  distribution of image vectors could be simplified
  to linear pattern.

12
Model-based face recognition

• The model-based face recognition scheme is aimed
  at constructing a model of the human face, which
  is able to capture the facial variations.
• Model-based approaches derive distance and
  relative position features from the placement of
  internal facial elements (eyes, nose).
• Generally, a face model contains the information
  of shape and texture of the face.

13
Bunch Graph

• Human faces have a similar topological structure.
• Face can be structured by nodes located at some
  specific points and edges labeled with distance
  vectors, then a face graph is produced.
• Face Bunch graph is generated from a set of
  sample face images. The FBG serves as a general
  representation of a set of faces.
• The stacks of discs on a node contain a bunch of
  description of facial features.
• Each stack of discs called a jet represents an
  alternative of facial feature description.
• The edges are labeled with averages of distance
  vectors.

14
Elastic Bunch Graph Matching

• To recognize a new face by elastic bunch graph
  matching, the fiducial points are positioned so
  as to extract a graph, which maximize a graph
  similarity between this graph and the FBG.
• After the nodes has been located on the new face,
  the face can be recognized by comparing the
  similarity between that the graph of this face
  and graphs of every face store in the FBG.

15
An Active Appearance Model

• The AAM is constructed based on a set of labeled
  images, where landmark points are marked on each
  example face at key positions to describe the
  facial features.
• Models are combined together by using Linear
  Analysis methods such as PCA.
• The vector of parameters for the combined model
  is controlling the shape and texture of models.
• AAM fitting is applied to seek a set of model
  parameters that best represents the test face
  image.
• The goal of recognition is to find the best match
  between the test parameter vector and training
  parameter vector.

16
3D Morphable model

• Human face is a surface lying in the 3D space.
  Thus, the 3D model is more suitable for
  representing faces,
• 3D model has stronger ability to minimize the
  problems of head pose, illumination.
• 3D morphable model is extended from 2D morphable
  model - AAM.
• Similar recognition methods on 2D morphable model
  can be improved and applied on 3D model as well.

17
Face Databases and Performance Evaluation

• There are about 28 face databases available
  currently, such as FERET, XM2VTS and UMIST etc.
• How to choose the suitable database based on the
  task given and the algorithm needs.
• FERET is a poplar face image database, which
  contains 1564 sets of images for a total of
  14,126 images that includes 1199 individuals and
  365 duplicate sets of images.
• False Acceptance / False Rejection and Equal
  Error Rate are scores to evaluate the similarity
  between a test pattern and a template.
• The Face Recognition Vendor Test was started
  from 2000 based on the FERET database. The
  database used in FRVT was extended 2 years later
  in FRVT2002.

18
References

• W. Zhao, R. Chellappa, A. Rosenfeld, P.J.
  Phillips, Face Recognition A Literature Survey,
  ACM Computing Surveys, 2003, pp. 399-458
• X. Lu, Image Analysis for Face Recognition,
  personal notes, May 2003, 36 pages
• H. Moon, P.J. Phillips, Computational and
  Performance aspects of PCA-based Face Recognition
  Algorithms, Perception, Vol. 30, 2001, pp.
  303-321M.A. Turk, A.P. Pentland, Face
  Recognition Using Eigenfaces, Proceedings of the
  IEEE Conference on Computer Vision and Pattern
  Recognition, 3-6 June 1991, Maui, Hawaii, USA,
  pp. 586-591
• M.A. Turk, A.P. Pentland, Face Recognition Using
  Eigenfaces, Proceedings of the IEEE Conference on
  Computer Vision and Pattern Recognition, 3-6 June
  1991, Maui, Hawaii, USA, pp. 586-591
• C. Liu, H. Wechsler, Comparative Assessment of
  Independent Component Analysis (ICA) for Face
  Recognition, Proc. of the Second International
  Conference on Audio- and Video-based Biometric
  Person Authentication, AVBPA'99, 22-24 March
  1999, Washington D.C., USA, pp. 211-216
• A. Pentland, B. Moghaddam, T. Starner, View-Based
  and Modular Eigenspaces for Face Recognition,
  Proceedings of the IEEE Conference on Computer
  Vision and Pattern Recognition, 21-23 June 1994,
  Seattle, Washington, USA, pp. 84-91
• K. Etemad, R. Chellappa, Discriminant Analysis
  for Recognition of Human Face Images, Journal of
  the Optical Society of America A, Vol. 14, No. 8,
  August 1997, pp. 1724-1733
• W. Zhao, R. Chellappa, A. Krishnaswamy,
  Discriminant Analysis of Principal Components for
  Face Recognition, Proc. of the 3rd IEEE
  International Conference on Face and Gesture
  Recognition, FG'98, 14-16 April 1998, Nara,
  Japan, p. 336
• A.M. Martinez, A.C. Kak, PCA versus LDA, IEEE
  Trans. on Pattern Analysis and Machine
  Intelligence, Vol. 23, No. 2, 2001, pp. 228-233
• J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos,
  Face Recognition Using LDA-Based Algorithms, IEEE
  Trans. on Neural Networks, Vol. 14, No. 1,
  January 2003, pp. 195-200
• C. Liu, H. Wechsler, Evolutionary Pursuit and Its
  Application to Face Recognition, IEEE Trans. on
  Pattern Analysis and Machine Intelligence, Vol.
  22, No. 6, June 2000, pp. 570-582
• L. Wiskott, J.-M. Fellous, N. Krueuger, C. von
  der Malsburg, Face Recognition by Elastic Bunch
  Graph Matching, Chapter 11 in Intelligent
  Biometric Techniques in Fingerprint and Face
  Recognition, eds. L.C. Jain et al., CRC Press,
  1999, pp. 355-396
• M.-H. Yang, Kernel Eigenfaces vs. Kernel
  Fisherfaces Face Recognition Using Kernel
  Methods, Proc. of the Fifth IEEE International
  Conference on Automatic Face and Gesture
  Recognition, 20-21 May 2002, Washington D.C.,
  USA, pp. 215-220
• .-H. Yang, Face Recognition Using Kernel Methods,
  Advances in Neural Information Processing
  Systems, T. Diederich, S. Becker, Z. Ghahramani,
  Eds., 2002, vol. 14, 8 pages
• T.F. Cootes, K. Walker, C.J. Taylor, View-Based
  Active Appearance Models, Proc. of the IEEE
  International Conference on Automatic Face and
  Gesture Recognition, 26-30 March 2000, Grenoble,
  France, pp. 227-232
• V. Blanz, T. Vetter, Face Recognition Based on
  Fitting a 3D Morphable Model, IEEE Transactions
  on Pattern Analysis and Machine Intelligence,
  Vol. 25, No. 9, September 2003, pp. 1063-1074
• B. Moghaddam, J.H. Lee, H. Pfister, R. Machiraju,
  Model-Based 3D Face Capture with
  Shape-from-Silhouettes, Proc. of the IEEE
  International Workshop on Analysis and Modeling
  of Faces and Gestures, AMFG, 17 October 2003,
  Nice, France, pp. 20-27
• J. Lee, B. Moghaddam, H. Pfister, R. Machiraju,
  Finding Optimal Views for 3D Face Shape Modeling,
  Proc. of the International Conference on
  Automatic Face and Gesture Recognition, FGR2004,
  17-19 May 2004, Seoul, Korea, pp. 31-36