Sin t

1

• Coded Structured Light
• Course
• Stereoscopy vs. Structured Light
• Coded Structured Light
• Principle Survey

• Coded Structured Light
• Course
• Stereoscopy vs. Structured Light
• Coded Structured Light
• Principle Survey

2
II/TAP 3D Perception
The camera behaviour?
Camera Modelling Identify the set of parameters
which determines the geometry and optical
characteristics of an approximated
camera. Camera Calibration Compute the value of
the parameters for a given sensor.
retinal
Z
image
r
co-ordinate
co-ordinate
Zi
system (mm.)
Y
system (pixels)
O
r
i
Xi
Yi
Image Centre or Principal Point
X
r
Focal distance
O
r
(u
, v
)
3D point
0
0
P
P
Focal point
f
World
d
Y
c
co-ordinate system
P
X
Y
Z
u
w
Image plane
c
w
X
c
Retinal plane
Z
Observed projection
Ideal projection
w
Camera
O
c
O
co-ordinate system
w
K
3
II/TAP 3D Perception
Obtaining 3D information?
M
Y
w
3D point

• From m, only the optical ray can be computed.
• We need more information to determine M.

?
X
w
optical ray
Z
w
Z
C
f
I
m
O
Y
X
Captured Image
4
II/TAP 3D Perception
Obtaining 3D information?

• We need at least two cameras.
• A 3D object point has three unknown co-ordinates.
• Each 2D image point gives two equations.

3D point
Y
M
w
X
w
Z
w
Z
Z
C
C
f
m
f
Y
Only a single component of the second point is
needed.
I
O
m
O
Y
I
X
X
Captured Image
Captured Image
5
II/TAP 3D Perception
3D Reconstruction is obtained !
Top View
Lateral View
6
II/TAP 3D Perception
The Correspondence Problem

• Given an image point m on an image plane,we have
  to search for its correspondence point m on the
  other image plane to obtain the co-ordinates of
  the 3D object point M .
• Problems
• Surface occlusions, vanishing points, cameras
  scope.
• Consequences
• Points without homologues.
• Points with multiple homologues.
• Solutions
• Matching from the epipolar geometry.
• Matching from disparity.
• Matching from similarity.
• ...

7
II/TAP 3D Perception
The Epipolar Geometry

• The searching space is reduced to a single line.

3D point
Epipole
Epipole
2D point
Epipolar line
2D point
Epipolar line
Image plane of the second camera
Image plane of the first camera
8
II/TAP 3D Perception
The Epipolar Geometry
Area 2
Epipole
Area 1
Epipole
Epipolar lines
Epipolar lines
Correspondence points
Zoom Area 2
Zoom Area 1
Epipolar geometry of Camera 2
Epipolar geometry of Camera 1
9
II/TAP 3D Perception
The Epipolar Geometry Matching Problems

• Scene Geometry
• Unknown Order of Projections.
• Points with multiple homologues.
• Surface occlusions Out of camera scope
• Points without homologue.

Surface occlusion
Point without homologue
10
II/TAP 3D Perception
Structured Light

• Principle Based on the relation between a
  single camera and a light source which projects a
  known pattern on the measuring scene.
• Only the points illuminated by the light source
  are captured by the camera.

• Advantages
• Easy segmentation. Depending on the light
  source.
• Correspondence problem reduced. Depending on the
  pattern shape.
• Able to reconstruct continuous surfaces.
• Constraints
• Light dependent.

11
II/TAP 3D Perception
Structured Light Techniques

• Stripe patterns
• Correspondence problem among slits.
• No scanning.
• Grid, multiple dots
• Correspondence problem among all the imaged
  segments.
• No scanning.

• Single dot
• No correspondence problem.
• Scanning both axis.
• Single slit
• Correspondence problem among points of the same
  slit gt orthogonal translation of the light
  projector with respect to the slit.
• Scanning the axis orthogonal to the slit.

12
II/TAP 3D Perception
Coded Structured Light

• Principle Based on an unique codification of
  each token of light projected on the scene. When
  the token is imaged by the camera this
  decodification permits to know where it comes
  from and solve the correspondence problem.

13
II/TAP 3D Perception
Pattern Crossing Points
Image Crossing Points
Pattern Projector Frame
Camera Image Plane
Image segmentation
Searching the correspondence point
3D Reconstruction
14
II/TAP 3D Perception
Coded Structured Light Classification
Projected Light Dependence

• Binary
• Grey Levels
• Colour

15
II/TAP 3D Perception
Coded Structured Light Techniques

• Le Moigne - Waxman 1984
• Pattern placed with single vertical displacement
  from camera.
• Vertical slits used as guides to search the
  dots.
• Dots are used to decode the horizontal lines.
• Both axis coded / Dynamic / Binary / Absolute

• Posdamer - Altschuler 1981
• The projection of n patterns are needed to code
  2n columns.
• Projection of an initial pattern without
  codification to extract dot positions.
• Column Coded / Static / Binary / Absolute
• Minou 81, Inokuchi 84. The same pattern
  proposal.
• Sato 86. Better segmentation, positive
  negative images.
• Mundy 87. Sensor implemented in hardware.
• Hattori 95. Low heat irradiance. Semiconductor
  Laser.
• Muller 95. Re-implementation.

16
II/TAP 3D Perception
Coded Structured Light Techniques

• Carrihill - Hummel 1985
• Two shots are needed
• First, constant illuminated pattern.
• Second, the linear wedge grey pattern.
• Column decodification from image substraction.
• Column Coded / Static / Grey / Absolute

• Maruyama - Abe 1993
• Multiple vertical slits with random cuts.
• Matching from slit end points by using epipolar
  geometry.
• Column Coded / Dynamic / Binary / Absolute

17
II/TAP 3D Perception
Coded Structured Light Techniques

• Morita - Yakima - Sakata 1988
• Initial projection of a whole illuminated dot
  matrix to extract dot position.
• Window coded pattern.
• Column Coded / Static / Binary / Absolute
• Lavoie 96. A grid pattern with random binary
  dots in the cross-points.

• Vuylsteke - Oosterlinck 1990
• Chess-board pattern projection with coded
  squares.
• Window coded pattern.
• Column Coded / Dynamic / Binary / Absolute
• Pajdla 95 Re-implementation.
• Ito 95 A three grey level checkerboard pattern.

18
II/TAP 3D Perception
Coded Structured Light Techniques

• Boyer - Kak 1987
• Multiple coloured vertical slits.
• Codification from slit colour sequence.
• Column Coded / Dynamic / Colour / Absolute
• Monks 93 Utilisation of the same pattern for
  speech interpretation.
• Chen 97 Unique codification and colour
  improvement.

• Tajima - Iwakawa 1990
• Projection of a rainbow pattern.
• Monochromatic cameras Two images were captured
  using two different camera filters.
• Column Coded / Static / Colour / Absolute
• Smutny 96 Re-implementation.
• Geng 96 Single RGB grabber.

19
II/TAP 3D Perception
Coded Structured Light Techniques

• Boyer - Kak 1987 Slit based projection
• Single axis codification.
• Column coded by groups of 4 slits.
• Problem of matching.
• Improve
• Both axis codification.
• Project some additional information in order to
  identify if two slits are neighbours or not. Þ
  Grid pattern projection determine the
  neighbourhood by slit continuity.

20
II/TAP 3D Perception
Coded Structured Light Techniques

• Wust - Capson 1991
• Periodical pattern made by shifting and merging
  red, green and blue sinusoidal intensity
  patterns.
• Column Coded / Dynamic / Colour / Periodical
• Hung 93. Single grey level sinusoidal pattern.

• Griffin - Narasimhan - Yee 1992
• Mathematical study to obtain the largest
  codification matrix from a fixed number of
  colours.
• Dot position coded by the colour of its four
  neighbours.
• Both axis coded / Static / Colour / Absolute
• Davies 96 Re-implementation.

21
II/TAP 3D Perception
Coded Structured Light Techniques

• Griffin et al. 1992 Dot matrix projection
• Both axis codification.
• Dot position coded by the colour of itself and
  its four neighbours.
• (C E N W S)
• Horizontal and vertical neighbours coded by
  using the same colours.
• Depending on the scene, sometimes its difficult
  to determine the neighbours.
• Improve

• Connect the dots with slits in order identify
  the neighbours uniquely.

• Horizontal and Vertical axis coded independently
  Þ Projector placed without positional
  constraint.

22
II/TAP 3D Perception
Coded Structured Light Techniques
Griffin 92
Wust 91
Tajima 90
Boyer 87
Vuylsteke 90
Morita 88
Carrihill 85
Moigne 84
Posdamer 81
Maruyama 93
Summary
Single Axis
Codification
Both Axis
Static
Temporal
Dynamic
Binary
Light
Grey
Colour
Periodical
Depth
Absolute
23
II/TAP 3D Perception
Coded Structured Light Techniques
What is it required?
How can it be done?
Easy Segmented
Projection of a Grid
Measurement of Dynamic Scenes
Pattern coded in a single projection
Accurate Results
Both axes codification
Without positional constraint with respect to
the camera
Vertical and Horizontal slits coded independently
Use of colour
Each slit with its two neighbours exits only
once in the pattern
Without depth dependence
24
II/TAP 3D Perception
Coded Structured Light Techniques
As an example, we have used p 3 P
1,2,3 Slit colour sequence 3313213112312212111
3323222333 Horizontal 1 red, 2 green, 3
blue. Vertical 1 magenta, 2 cyan, 3
yellow.

• Colours well-spaced in the chromatic space.
• Maximum 29 slits with p 3 Þ Grid of 29 x 29
  slits.
• Slits are constantly spaced in a 512 x 512
  image.
• Resolution increase
• Increasing the p basis.
• Reducing the space between slits.

25
II/TAP 3D Perception
A way of obtaining a sequence of uniquely encoded
slits

• Given P1,2,...,p set of colours.
• We want to determine Ss1,s2,...sn sequence of
  coloured slits.
• Node ijk Î
• Number of nodes p3 nodes.
• Transition ijk Ù rst Þ j r, k s

Example p 2 Path
(111),(112),(122),(222),(221),(212),(121),(211).
Slit colour sequence111,2,2,2,1,2,1,1 Þ
Maximum 10 slits.
26
II/TAP 3D Perception
System Testing
Camera Image Plane
3D Reconstruction
Pattern Projector Frame
Smooth surface Reconstruction
27
System Testing
28
II/TAP 3D Perception
Accuracy Results
Real Scene
Reconstructed Scene
Average discrepancy values Xw 0.728 mm. (0.31
) Yw 0.624 mm. (0.27 ) Zw 0.465 mm. (0.20 )
Maximum discrepancy values Xw 1.15 mm. (0.50
) Yw 1.43 mm. (0.62 ) Zw 0.68 mm. (0.29 )
29
II/TAP 3D Perception
Conclusions

• Structured light has been presented as a
  technique which permits to reduce the
  correspondence problem. However, structured light
  can not solve the problem of multiple matching.
• Coded structured light has been presented as a
  technique to solve completely the matching.
• Proposal of new coded structured light
  challenges.
• The pattern coded with respect to both axis which
  permits to obtain more accurate results.
• The pattern is coded by a single projection which
  enables to obtain 3D information with a single
  shot, allowing the system to measure moving
  scenes.
• Vertical and horizontal slits are coded
  independently which permits the projector to be
  placed without positional constraints with
  respect to the camera.