The%20CSU%20Face%20Identification%20Evaluation%20System:%20Its%20Purpose,%20Features%20and%20Structure%20David%20S.%20Bolme,%20J.%20Ross%20Beveridge,%20Marcio%20Teixeira%20and%20Bruce%20A.%20Draper%20Computer%20Science,%20Colorado%20State%20University

1
The CSU Face Identification Evaluation
SystemIts Purpose, Features and
StructureDavid S. Bolme, J. Ross Beveridge,
Marcio Teixeira and Bruce A. DraperComputer
Science, Colorado State University

• 3rd International Conference on Computer Vision
  Systems - ICVS 2003

2
Goals of the CSU Face Recognition Evaluation Work

• Baseline/control Face Recognition algorithms.
• Four algorithms selected from FERET 96/97 study.
• PCA, Eigenfaces (Turk and Pentland, MIT)
• PCALDA, (Zhao et. al., Maryland)
• Bayesian Image diff. Classifier, (Moghaddam et.
  al., MIT)
• Elastic Bunch Graph (Okada, et. al., USC)
• Reference implementations in ANSI C.
• CSU Face Identification Evaluation System
• Statistical methodology for studying algorithms.
• Parametric and Nonparametric methods
• Standardized protocols and associated scripts.
• Determine critical factors that influence
  performance.

3
Obtaining the CSU Face Identification Evaluation
System

• The Evaluation of Face Recognition Algorithms
  Website.
• First release of code on March 1, 2001
• Current code release,
• Version 4.0,
• October 31, 2002
• Over 1,500 downloads of Version 4.0 through March
  2003
• Users Guide is included and also available
  separately.

http//www.cs.colostate.edu/evalfacerec/algorithms
4.html
4
CSU Face Identification Evaluation System Users
Guide

• Installation
• Testing the system
• Scripts,scrapshots
• System Overview
• Image Formats
• Distance Files
• Image Preprocessing
• Algorithms
• PCA,
• PCALDA,
• BIC
• Analysis
• Cumulative Match Curves
• Error bars distributions

This ICVS 2003 Paper overlaps parts of the Users
Guide
5
System Overview
Preprocessing
Normalization
Training
Subspace Training
Bayesian Training
Testing
Subspace Project
Bayesian Project
Analysis
Rank Curve Testing
Permutation Testing
Standard Cumulative Match Curves
Probability Distribution for Recognition Rate
6
Image Preprocessing

• Integer to float conversion
• Converts 256 gray levels to single-floats
• Geometric Normalization
• Aligns human chosen eye coordinates
• Masking
• Crop with elliptical mask leaving only face
  visible.
• Histogram Equalization
• Histogram equalizes unmasked pixels 256 levels.
• Pixel normalization
• Shift and scale pixel values so mean pixel value
  is zero and standard deviation over all pixels is
  one.

Refinement of NIST preprocessing used in FERET.
7
The csuSubspace modulePCA and PCALDA
Training

Training images
Eigenspace
Combined space (PCALDA)
Testing
Distance Matrix

PCALDA space projection
8
Bayesian Image difference Classifier Take
Difference of Images

• Classify difference image as either
• Intrapersonal from same subject
• Extrapersonal from different subjects

Intrapersonal Example
-

Extrapersonal Example
-

9
Bayesian Image difference ClassifierTraining
Uses csuSubspace Module
Extrapersonal
All Training Images
csuSubspaceTrain
Extrapersonal PCA Subspace
csuMakeDiffs
csuSubspaceTrain
Intrapersonal PCA Subspace
Intrapersonal
10
Bayesian Image difference ClassifierTesting
uses csuBayesianProject
Probe Gallery Images
Extrapersonal PCA Subspace
Distance Matrix
CsuBayesianProject
Intrapersonal PCA Subspace
11
Evaluation Methodology and Tools
Two Distinct Questions
Generalized Linear Model Example Mixed Effects
Logistic Regression with Repeated Measures on
People.

1. Is an observed difference in performance
   significant?

• McNemars Test.
• Monte Carlo Inference.

Version 4.0

• What covariates, and combinations of covariates,
  most influence performance?
• And how much?

Monte Carlo Inference Example Sample Recognition
Rate Probability Distribution created by
perturbing probe gallery choice.

• Monte Carlo Inference.
• Generalized Linear Models.

McNemarsTest Tally when one algorithm succeeds
and the other fails.
Covariates covers both features of algorithms and
of people
Weak
Strong
Power
12
Training, Probes, Galleries, What Varies?
Fixed Throughout Study
F
Training
Probes
Gallery
Varied, i.e. randomly sampled
V
F
F
F
Essentially FERET 1996/97
F
F
V
Micheals Boult CVPR 2001
F
V
F
F
V
V
CSU PCA vs. PCALDA Analysis
F
F
V
F
V
V
V
F
V
CSU PCALDA Configuration Analysis
V
V
V
13
Producing Cumulative Match Curves
14
Producing Sample Distributions

• Compare PCA and PCALDA.
• Distance Measures L1, L2, Mah. Angle (PCA), Soft
  L2 (PCALDA).
• Methodology Monte Carlo Sampling of
  Probe/Gallery.

Balanced Sampling
Id. 1 2 3 4
114 P G
67 P G
53 P G
145 P G
6 G P
154 G P
71 G P
98 G P

99 G P
CVPR 2001 citation.
15
PCA vs. PCALDA Confidence Intervals
Sample Probability Distribution for PCA at rank 1
using Mahalanobis Distance.
0.12
0.10
0.08
Probability
0.06
0.04
0.02
0.00
0.50
0.52
0.54
0.56
0.58
0.59
0.61
0.63
0.65
0.67
0.69
0.71
0.73
0.75
16
Tabular Output from csuPermute
17
PCA vs. PCALDAComparing Distance Measures

• Distance Measure Matters
• PCA favors Mahalanobis Angle
• PCALDA, Soft and Angle Similar
• Cumulative Match with Error Bars
• Distance choice more important than subspace.

18
Current Research FERET Subject Covariates
Covariates for 2,974 Images, 1,209 Subjects
19
FERET Covariates Results (Preliminary!)
Harder to Recognize
Easier to Recognize
Change in Similarity Measure
20
Conclusion

• Release 4.0 Contains
• Three algorithms PCA, PCALDA, BIC.
• Cumulative match curve and probe gallery
  permutation tools.
• Scripts for common experiments, including
  standard FERET.
• Supported platforms include
• Code is ANSI C Unix, Windows,
• Turn-key scripts and code tested on Linux,
  Solaris, Darwin.
• Over 1,500 downloads since October 31, 2002.
• Related papers on web site.
• Near Future - Release 5.0
• Elastic Bunch Graph Matching (USC FERET).
• Data Preparation for Generalized Linear Models.
• PCALDA Configuration and FERET Subject Covariate
  Study.

21
The End
22
Help for csuPreprocesNormalize
23
Help for SubspaceTrain
24
Help for csuSubspaceProject
25
Help for csuMakeDiffs First step in Bayesian
Algorithm
26
Help for csuBayesianProject
27
Help for csuAnalysis Tools