REAL TIME FACE DETECTION

1
REAL TIME FACE DETECTION

• Justin Rilling
• Pooja Mhapsekar
• Moinuddin Sayed
• Ogom J Obinor

2
Introduction

• The idea is to be able to detect faces appearing
  in an image.
• The faces can be of different sizes and
  orientations.
• The motivation behind implementing this on an
  FPGA is that image processing is inherently
  parallel in nature and lends itself well to an
  FPGA.

3
Viola and Jones Detector

• Key Contributions
• 1. Integral Image fast computation of features
  used by the detector.
• 2. Combines complex classifiers in cascade
  focus on facial features.

4
Integral Image

• Major contribution of the Viola and Jones
  detector.
• Summation of pixel values of the original image.
• Value at location (x,y) sum of values of pixels
  above and to the left of (x,y).

• Whole image converted to integral image and a
  window buffer used to scan the entire image.

5
Haar Classifiers

• Classifiers are composed of 2-3 rectangles.
• Weights and sizes associated with features
  obtained through AdaBoost.
• Classifier sum ? (area weights)

• Integral image makes area calculation easy. Area
  L4L3L2L1.
• Several classifiers compose a stage.

6
Stage

• Sum of all Haar feature classifiers compared with
  the stage threshold.
• Multiple stages, each stage has different number
  of classifiers.
• Threshold obtained from AdaBoost algorithm.

7
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
8
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
426
542
Right Value
0.8378106
0
Stage Sum
Stage Threshold

lt
0.8226894
0.0
Left Value
0.0337941
1
9
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
426
542
Right Value
0.8378106
0
Stage Sum
Stage Threshold

lt
0.8226894
0.0337941
Left Value
0.0337941
1
10
Face Detection Example
Stage Number
0
Classifier Number
1
Classifier Sum
Classifier Threshold
lt
887
2045
Right Value
0.7488812
0
Stage Sum
Stage Threshold

lt
0.8226894
0.0337941
Left Value
0.1514132
1
11
Face Detection Example
Stage Number
0
Classifier Number
1
Classifier Sum
Classifier Threshold
lt
887
2045
Right Value
0.7488812
0
Stage Sum
Stage Threshold

lt
0.8226894
0.1852073
Left Value
0.1514132
1
12
Face Detection Example
Stage Number
0
Classifier Number
2
Classifier Sum
Classifier Threshold
lt
349
568
Right Value
0.6374819
0
Stage Sum
Stage Threshold

lt
0.8226894
0.1852073
Left Value
0.0900493
1
13
Face Detection Example
Stage Number
0
Classifier Number
2
Classifier Sum
Classifier Threshold
lt
349
568
Right Value
0.6374819
0
Stage Sum
Stage Threshold

lt
0.8226894
0.2752566
Left Value
0.0900493
1
14
Face Detection Example
Stage Number
0
Classifier Number
2
Classifier Sum
Classifier Threshold
lt
349
568
Right Value
0.6374819
0
Stage Sum
Stage Threshold

lt
0.8226894
0.2752566
Left Value
0.0900493
1
Window Buffer Failed Stage 0
15
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
16
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
17
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
18
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
19
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
20
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
21
Face Detection Example
Stage Number
0
Classifier Number
0
Classifier Sum
Classifier Threshold
lt
0.0
0.0
Right Value
0.0
0
Stage Sum
Stage Threshold

lt
0.0
0.0
Left Value
0.0
1
22
Top Level Design
Get Scaled Image
Get Integral Image
Window Buffer (21x21)
Get Image
Draw Rectangle If face is det.
End of Image?
Face Detection Pipeline
YES
Monitor
NO
23
Face Detection Pipeline
24
Original End Product
USB Webcam
FPGA
Display
USB
DVI
25
Updated End Product
FPGA
Display
DVI
Digital Interface

• 1/3 Color Camera Mod C3188A-6018
• Supports several standard image data formats
  including YCrCb 422
• 640 x 480 resolution
• 5 V signaling
• 30 fps

26
Updated End Product
XC95144XL CPLD
FPGA
3.3 V Signaling
5 V Signaling
DVI

• 1/3 Color Camera Mod C3188A-6018
• Supports several standard image data formats
  including YCrCb 422
• 640 x 480 resolution
• 5 V signaling
• 30 fps

Display
27
DVI Controller
28
Questions ???

• References
• P. Viola and M. Jones, Robust real-time object
  detection, International Journal of Computer
  Vision, 57(2), 137-154, 2004.
• Junguk Cho, Shahnam Mirzaei, Jason Oberg, Ryan
  Kastner, FPGA-Based Face Detection System Using
  Haar Classifiers, Proceeding of the ACM/SIGDA
  International Symposium on Field Programmable
  Gate Arrays, 103-112, 2009.

29
OpenCV Program Lena.jpg
30
OpenCV Program ER.jpg
31
OpenCV Program ER.jpg

• Window Buffer (2, 152, 122)

32
Face Detection Pipeline
33
OpenCV Program ER.jpg

• Window Buffer (3, 165, 20)

34
Face Detection Pipeline
35
Questions ???