Algorithm Design

1
Camera Phone Color Appearance Utility Finding a
Way to Identify Color
Phillip Lachman Robert Prakash Elston Tochip
2
Outline

• Motivation
• Goal
• Methodology
• Image Scaling via Edge Detection
• Color Identification
• Color Selection Differentiation
• Results
• Lessons Learned
• Future Work

3
Motivation

• Phones becoming the portal able digital platform
  for variety of imaging applications i.e.
  pictures, video, organizers etc.
• Approximately 10 million blind people within the
  U.S.
• 55,200 legally blind children
• 5.5 million elderly individuals
• http//www.afb.org/Section.asp?sectionid15num
• Color blind people within the U.S.
• 8 of males
• 0.4 - 2 females
• http//www.otal.umd.edu/UUPractice/color/

4
Goal

• To develop a software application that will be
  able to accomplish the following
• 1) Receives a camera quality image
• 2) Identifies the predominant color(s) regions
  within the image
• 3) Estimates the color name for the predominant
  region
• 4) Audibly transmits the predominant color to the
  user

5
Locating the Target
6
General Guidelines and Suggestions

• Use a White Card
• Provides a white color to baseline lighting
  conditions
• Required for computing color of target
• Suggested by fellow classmates and Bob Dougherty
• Detecting the White Card
• Use an Edge Detection Algorithm
• Many Image Processing Edge detection methods
  available
• Identify edges by computing changes in gradient
  around pixels.
• Chose Canny Edge Detection algorithm
• Fundamentally easy to understand and implement

Edge Detection
7
Finding the Target Discrimination

• How does the algorithm discriminate the target
  being photographed?
• Background clutter and scenery complicate the
  image
• Discrimination Solution
• The White Card Scope
• Use the rectangular white card with a square
  target hole to allow object color through
• Use Edge Detection image processing algorithms to
  find the white card
• Find the white card, find the target!

Background
Target Color
8
Finding the Target Discrimination Cont

• The White Card Problem
• White backgrounds or light color backgrounds
  cause edge detection problems in Canny Algorithm

After Edge Detection
Original Image
Where is the card??
9
Finding the Target Discrimination Cont

• The White Card Problem Cont.
• Adding a Black Outline to the card edges and
  target hole greatly improve detection!

Original Image
After Edge Detection
Theres the card!!
10
Finding the Target Aiming the Camera

• How does a blind person AIM the camera to take a
  picture of the target?
• Photographs may NOT include target
• Photographing target too close may not allow
  enough lighting to determine color
• Aiming Solution- White Card Holder
• Use a phone attachment which holds the white card
  AND attaches to phone camera
• Guarantees white card and target in the camera
  Field of View
• Guarantees camera is not directly on top of the
  object, providing ample lighting for color
  detection

11
Finding the Target Aiming the Camera

• Additional Benefits of Card Holding Device
• Fixes Orientation of the card
• Chose to have card positioned vertically with
  edges parallel to photo edges
• Simplifies algorithm detection, increasing speed
• Removes excess background scenery
• Device maintains a fixed 6-8 inches between
  camera and white card
• Scene is dominated by white card and maximizes
  number of pixels covering the target

12
Finding the Target Examples with and without
device
13
Finding the Target Edge Detection Algorithm

• Phase 1 Blurring and Sharpening Edges
• Preprocess Images to blur and eliminate noisy
  pixels
• Apply a 3x3 Laplacian Matrix Kernel to resulting
  image
• Kernel is an approximation of the second
  derivative, highlighting changes in intensity
• Matrix
• Adding results of Gaussian Blurring and Laplace
  yields image with cleaner and more distinct lines

-1 -1 -1
-1 8 -1
-1 -1 -1
14
Edge Detection Blurring and Sharpening
Original Image
After Smoothing and Laplace
Applying Laplace removes Noise and Smoothes lines
15
Finding the Target Edge Detection Algorithm

• Phase 2 Apply Canny Edge Detection Algorithm
• Step 1 Applies Gaussian smoothing in 2
  dimensions to the image via convolution
• Size of the mask 20x20 with a sigma 5
• Step 2 Compute the resulting gradient of the
  intensities in the image
• Step 3 Threshold the norm of the gradient image
  to isolate edge pixels

16
Edge Detection Applying Canny Edge Detection
Algorithm
Original Image
After Canny and Threshold
17
Finding the Target Edge Detection Algorithm

• Phase 3 Finding the edges in the photo
• Step 1 Recursively search, row by row, from the
  outer left/right edges of the image towards the
  center.
• Search for 1 quarter length from right/left side

18
Finding the Target Edge Detection Algorithm

• Phase 3 Cont
• Step 2 Bin and compute outer edges based on
  which values are closest to the center
• Find left/right edges based on bin having AT
  LEAST 10 of the total pixels available on the
  each side
• Step 3 Compute Top and Bottom Edge
• Compute the average row at which both the left
  and right edges begin/end
• Value gives rough estimate of top/bottom of white
  card

First bin From Center Exceeding 10
First bin From Center Exceeding 10
Right Side
Left Side
19
Finding the Target Target Hole Detection
Algorithm

• Outer Target White Card edges Located
• Proceed to Locate INNER Target Hole edges
• Phase 4 Identifying Inner Target Hole Edges
• Step 1 Crop Canny Thresholded image to
  dimensions obtained for outer edge
• Step 2 Perform recursive row by row outside to
  inside search until a high threshold is found on
  both sides.
• Step 3 Bin and compute left/right edges as
  before
• Step 4 Compute Top and Bottom edges as before

20
Finding the Target Output to Color Detection

• Phase 5 Compute overall Target Hole Position in
  ORIGINAL image
• Sum up inner and outer edge values computed
  previously
• Crop the original image to these dimensions and
  output to Color Detection model

21
Color Detection Original Idea

• Keep it simple
• Use brightest point on white card as white point.
• Normalize R,G and B separately.
• Good results, slight reddish tinge.

22
Color Detection Gray World

• Use Gray world theory
• Normalize means of RGB to 128
• Results slightly better in low lighting
  conditions, but less effective under good
  lighting.

23
Color Identification Original Idea

• Keep it simple Bin using RGB
• Problem
• No clear grouping
• Small changes in one value, changes color
  dramatically
• Solution
• Attempt to identify groups by using max of R, G
  and B values.
• Still contained overlaps

24
Color Identification CIE- Lab

• Convert RGB to CIELab
• Benefits
• Device independent.
• Problems
• Conversion formulas complicated and processor
  intensive.
• Light source information is required.
• Solution use HSV

25
Color Identification Use HSV

• Convert RGB to HSV
• The HSV (Hue, Saturation, Value) model is a
  simple transformation from RGB.
• Hue, the color type (like red, blue, etc)
• ranges from 0-360
• Saturation, the "vibrancy" of the color
• Ranges from 0-100
• Value, the brightness of the color
• Ranges from 0-100

26
Color Identification RGB to HSV

• Equations used for conversion

27
Color Selection Differentiation

• Currently code identifies 24 colors based on HSV
  color system.
• Color identification is acceptable, but starts to
  fail in low lighting conditions.

28
Results
White
Light Green
White
Indian Red
29
Lessons Learned

• Edge Detection Think about the Big Picture
• User Feasibility is Critical
• If a soldier cannot aim a gun, how accurate is
  his shot?
• Simplicity is Essential
• Presetting card orientation led to efficiency and
  shortcuts for edge detection
• Slanted Orientation requires much more processing
  time and development
• Original code variants tried to and failed to
  account for all orientations

30
Lessons Learned

• Color Detection
• HSV is a compromise between simply binning on RGB
  values and conversion to Lab.
• Normalization using the white point more
  effective than gray world.
• Minimum level of lighting in required, since
  camera is low quality.

31
References

• http//robotics.eecs.berkeley.edu/mayi/imgproc/ca
  code.html
• http//homepages.inf.ed.ac.uk/rbf/HIPR2/canny.htm
  
• http//www.aquaphoenix.com/lecture/matlab10/page3.
  html
• http//en.wikipedia.org/wiki/Canny
• http//www.afb.org
• http//www.otal.umd.edu/UUPractice/color/
• Class notes on Color and jpeg tutorials

32
Future Work

• Implementing the processing onto an actual camera
  phone
• Decreasing the processing time to audibly deliver
  the color to the user
• Increasing color library
• Refining overall algorithm to distinguish more
  detailed backgrounds.
• Patches
• Patterns
• Color Designs