Poster Template

1
ReadMe! Facial Expression Recognition
System Eric DeFelice, Kevin Zhou, Yasemin
Ersoy Department of Electrical Engineering,
Stanford University
Detection of Facial Expression
Facial Recognition Technique
Use Haar Classifier 1 to detect and bound the
face.
1.
Find key points within the bounding box (i.e.
eyes, nose, eyebrows, mouth and upper lip).
2.
3.
Normalize key points w.r.t. the bounding box and
size. Average the key points over 3 frames.
analyze
4.
Calibrate subjects face over 100 frames, to set a
baseline
Figure 1. The face detection works by applying
several simply classifiers to a region
sequentially to reject all non-matched
classifiers. The basic classifiers are
decision-tree classifiers base on the features
above1. The features are detected by using
differences of boxes, and the convolution
kernel process can be done rapidly with a
pre-calculated integral image.   This procedure
then is repeated for all areas of the image by
sliding a search window across the image and
check every location using the classifier. To
detect features at different scales, scan
procedure is done several times at different
scales.   For the purpose of our project, we used
haarcascade_frontalface_alt2.xml as our
decision-tree classifiers, and this was trained
with examples of faces. And we have defined the
bounding box size from 40 40 to 200 200.
Facial expression is detected live through those
key points, and output on screen. The algorithm
may detect multiple expressions based on combined
inputs (i.e. raised eyebrows indicating surprise,
and smile indicating happiness). These inputs
are defined due to distance changes between
intrafeature key points.
5.
Calibration
Output
Facial expression recognition software that
attempts to determine a subjects real-time
expression through analysis of key point
locations relative to the subjects normalized
expression.
Figure 2. Example of key point detection using
various image processing techniques for eyebrow
detection. The output of the flandmark2 is used
as a priori information.
Key-point Labeling Optimization
Future Work
Algorithm changes lead to improvements in run
time and expression identification. Below is a
comparison between two approaches for eyebrow
detection.

• Improve key-point detection by using matched
  filtering with scale adjustment based on bounding
  box size
• Improve expression recognition using SVMs
  (support vector machines) and more accurate
  expression models
• Explore an approach using local binary patterns
  (LBPs) to detect facial features  and Adaboost to
  weight features with more powerful discriminabilit
  y.

Thresholding for any value greater then the mean
for that box
High-pass filter in Y direction for edge detection
Filter regions and label eyebrow
Figure 3. Graphs show the improvements in
key-point detection accuracy and speed. The
detection accuracy did not improve drastically
using the second algorithm, but the variance of
the error was decreased. The speed of the
detection was improved quite significantly
however.
Label regions.
Detect face in current video frame
Median filtering for noise reduction
Adaptive histogram equalization
Difference-of-box filtering in Y direction for
edge detection
Normalize image to 0 mean
References

1. OpenCV 2.45. OpenCV.org
2. M. Uricar, V. Franc and V. Hlavac, Detector of
   Facial Landmarks Learned by the Structured Output
   SVM, VISAPP '12 Proceedings of the 7th
   International Conference on Computer Vision
   Theory and Applications, 2012.
3. Huizhong Chen for algorithm support.

Threshold anything above a std. dev. from the mean
Filter regions and label eyebrow
Label regions.