Localization and Segmentation of 2D High Capacity Color Barcodes

1
Localization and Segmentation of 2D High Capacity
Color Barcodes
Gavin Jancke Microsoft Research, Redmond
Devi Parikh Carnegie Mellon University
2
Motivation
UPC Barcode
QR Code
Datamatrix
3
HCCB
Microsofts High Capacity Color Barcode
4
Application

• Uniquely identifying commercial audiovisual
  works such as motion pictures, video games,
  broadcasts, digital video recordings and other
  media

5
Goal

• Locate and Segment the barcode from consumer
  images

6
Overview

• Design specifications of Microsofts HCCB
• Approach
• Localization
• Segmentation
• Progressive Strategy
• Results
• Conclusions

7
Microsofts HCCB
4 or 8 colors Triangles String of colors
palette
8
Microsofts HCCB
9
Microsofts HCCB
10
Microsofts HCCB
11
Microsofts HCCB
Aspect ratio r
R rows
S (r1)R
S symbols per row
12
Approach
Thresholding
Orientation prediction
Barcode localization
Corner localization
Row localization
Symbol localization
Barcode segmentation
Color assignments
point inside the barcode is known
13
Localization Thresholding

• Identify thick white band and row separators
• Normalization
• Adaptive

14
Localization Orientation
summation
distance
-90
90
0
orientation
orientation
15
Localization Corners

• Rough estimates

whiteness mask
non-texture mask
combined mask
16
Localization Corners

• Gradient based refinement

17
Localization Corners

• Line based refinement

18
Segmentation Rows
Flip?
Summation
19
Segmentation Symbols
Number of symbols per row
q(S,E) Sq(samplesS,E)
Local search
20
Segmentation Colors
Palette
21
Observations

• Segmentation results given accurate localization
• Satisfactory
• Corner localization
• Unsatisfactory
• No one strategy works well on all images
• However
• (1) Errors of different strategies are
  complementary
• (2) Results are verifiable with decoder in
  the loop!

22
Progressive strategy

• Hence progressive strategy!
• Similar to ensemble of weak classifiers
• Or hypothesize-and-test
• Multiple strategies
• Rough gradient line, or rough line, or
  rough gradient, or rough alone
• Different values of threshold during rough corner
  detection
• Total 12
• Order of strategies

23
Results

• Dataset of 500 images
• Performance metric barcodes successfully
  decoded
• Decoder model Barcode successfully decoded if
  80 of symbols are correctly identified

24
Results
Allows for explicit trade-off between accuracy
and computational time
25
Results
26
Results
27
Results
28
Results
29
Results
30
Results
31
Results
32
Results
33
Results
34
Results
35
Conclusions

• 2D High Capacity Color Barcode (HCCB)
• Successful localization and segmentation of HCCB
  from consumer images
• Varying densities, aspect ratios, lighting, color
  balance, image quality, etc.
• Simple computer vision and image processing
  techniques
• Progressive strategy

36
Acknowledgements

• Microsoft Research
• Larry Zitnick
• Andy Wilson
• Zhengyou Zhang
• Carnegie Mellon University
• Advisor Tsuhan Chen

37

• Thank you!