3-D Computer Vision CSc 83020

1
3-D Computer VisionCSc 83020

• Camera Calibration

2
Camera Calibration

• Problem Estimate cameras extrinsic intrinsic
  parameters.
• Method Use image(s) of known scene.
• Tools
• Geometric camera models.
• SVD and constrained least-squares.
• Line extraction methods.

3
Camera Calibration
Perspective Equations
Feature Extraction
From Sebastian Thrun and Jana Kosecka
4
Perspective Projection, Remember?
O
X
x
f
Z
From Sebastian Thrun and Jana Kosecka
5
Intrinsic Camera Parameters

• Determine the intrinsic parameters of a camera
  (with lens)
• What are Intrinsic Parameters?
• (can you name 7?)

From Sebastian Thrun and Jana Kosecka
6
Intrinsic Parameters
O
Image center(ox, oy)
X
f
Z
From Sebastian Thrun and Jana Kosecka
7
Intrinsic Camera Parameters

• Intrinsic Parameters
• Focal Length f
• Pixel size sx , sy
• Image center ox , oy
• (Nonlinear radial distortion coefficients k1 ,
  k2)
• Calibration Determine the intrinsic parameters
  of a camera

From Sebastian Thrun and Jana Kosecka
8
Coordinate Frames
Camera Coordinate Frame
Zc
Pixel Coordinates
Yc
Intrinsic Parameters
Xc
Extrinsic Parameters
Image Coordinate Frame
Zw
Yw
World Coordinate Frame
Xw
9
Why Calibrate?
Image point
Image
Calibration relates points in the image to rays
in the scene
10
Why Calibrate?
Image point
x
y
z
Image
x
y
Calibration relates points in the image to rays
in the scene
11
Example Calibration Pattern
From Sebastian Thrun and Jana Kosecka
Calibration Pattern Object with features of
known size/geometry
12
Harris Corner Detector
From Sebastian Thrun and Jana Kosecka
13
Perspective Camera
r
r
(x,y,z)
(X,Y,Z)
Center of Projection
r (x,y,z) r(X,Y,Z)
xf X/Z yf Y/Z zf
r/fr/Z
f effective focal length distance of image
plane from O.
14
Extrinsic Parameters
T
PcR(Pw-T) Translation followed by rotation
15
Extrinsic Parameters (2nd formulation)
R same as before T different
PcR Pw T Rotation followed by translation
16
The Rotation Matrix
T
T
R R R R I gt R R Orthonormal
Matrix Degrees of freedom? 1 0 0 I 0 1 0 0 0 1
T
-1
17
Perspective Camera Model

• Step 1 Transform into camera coordinates
• Step 2 Transform into image coordinates

From Sebastian Thrun and Jana Kosecka
18
Intrinsic Parameters
19
Image and Camera Frames
Ycamera
Zcamera
Ximage
(xim, yim)
Xcamera
(ox,oy)
Yimage
20
Geometric Model
3D Point in Camera Coordinate Frame
XcT
x
ZcT
YcT
y
ZcT

• Transformation from Image
• to Camera Frame.
• (ox,oy,sx,sy)
• No distortion!

• Transformation from
• World to Camera Frame.
• Perspective projection
• (f, R, T)

Point in Camera Frame
21
Camera Calibration Issues

• Which parameters need to be estimated.
• Focal length, image center, aspect ratio
• Radial distortions
• What kind of accuracy is needed.
• Application dependent
• What kind of calibration object is used.
• One plane, many planes
• Complicated three dimensional object

22
Camera Calibration
Calibration object
Extracted features
23
Camera Calibration
Extract centers of circles
24
Basic Equations
25
Basic Equations
In the remaining slides x means xim and y means
yim
26
Basic Equations
27
Basic Equations
Extrinsic Parameters 1) Rotation matrix
R (3x3) 2) Translation vector T (3x1)

• Intrinsic Parameters
• fxf/sx, length in effective horizontal pixel
  size units.
• asy/sx, aspect ratio.
• (ox,oy), image center coordinates.
• Radial distortion coefficients.

Total number of parameters (excluding
distortion) ?
28
Basic Equations

• Assume that image center is known.
• Solve for the remaining parameters.
• Use N image points and their
• corresponding
• N world points

29
Basic Equations
(1)

• Assume that image center is known.
• Solve for the remaining parameters.
• Use N image points and their
• corresponding
• N world points

Here x means xim - ox and y means yim - oy
30
Basic Equations
(2)

• Assume that image center is known.
• Solve for the remaining parameters.
• Use N image points and their
• corresponding
• N world points

31
Basic Equations
(3)
32
Basic Equations
(3)
How would we solve this system?
33
Basic Equations
(3)
How would we solve this system? Rank of matrix
A? Solution up to a scale factor.
34
Singular Value Decomposition
Appendix A.6 (Trucco)
A m x n U m x m, columns orthogonal unit
vectors. V n x n , -//- D m x n ,
diagonal. The diagonal elements si are
the singular values s1gt s2gt gt sn gt
0
35
Singular Value Decomposition
Appendix A.6 (Trucco)

• Square A non-singular iff si ! 0
• For square A Cs1/sN is the condition number
• For rectangular A of non-zero si is the rank
• For square non-singular A
• For square A, pseudoinverse
• Singular values of A square roots of
• eigenvalues of and
• 7. Columns of U, V
• Eigenvectors of
• 8. Frobenius norm of a matrix

36
Singular Value Decomposition
Appendix A.6 (Trucco)

If rank(A)n-1 (7 in our case) then the solution
is the eigenvector which corresponds to the ONLY
zero eigenvalue. Solution up to a scale factor.
37
Solving for v
(3)
How would we solve this system SVD. Solution
Uknown scale factor ?? Aspect ratio a?
38
Solving for Tz and fx?
39
Solving for Tz and fx?
How would we solve this system?
40
Solving for Tz and fx?
How would we solve this system?
Solution in the least squares sense.
41
Camera Center
42
Camera Models (linear versions)
3D Point in World Coordinate Frame
x
y

• Transformation from
• World to Camera Frame.
• Perspective projection
• (f, R, T)

• Transformation from Image
• to Camera Frame.
• (ox,oy,sx,sy)
• No distortion!

Point in Camera Frame
43
Camera Models (linear versions)
Elegant decomposition. No distortion!
Homogeneous Coordinates
World Point (Xw, Yw,Zw)
Measured Pixel (xim, yim)
?
44
Camera Calibration Other method
Extracted features
Step 1 Estimate P Step 2 Decompose P into
internal and external parameters R,T,C
45
Camera Calibration Step 1
Extracted features
Each point (x,y) gives us two equations
46
Camera Calibration Step 1
Extracted features
Each corner (x,y) gives us two equations
47
Camera Calibration Step 1
2n
Extracted features
n points gives us 2n equations
48
Camera Calibration Step 1
2n
Extracted features
We need to solve
In the presence of noise we need to solve
The solution is given by the eigenvector with the
smallest eigenvalue of
49
Camera Calibration Step 1
The result can be improved through non-linear
minimization.
Extracted features
50
Camera Calibration Step 1
The result can be improved through non-linear
minimization.
Extracted features
Minimize the distance between the predicted and
detected features.