Stanford CS223B Computer Vision, Winter 200809 Lecture 4 Camera Calibration

1
Stanford CS223B Computer Vision, Winter
2008/09Lecture 4 Camera Calibration

• Professor Sebastian Thrun
• CAs Ethan Dreyfuss, Young Min Kim, Alex Teichman
  

2
Todays Goals

• Calibration Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software

3
Camera Calibration
Perspective Equations
Feature Extraction
4
Perspective Projection, Remember?
O
X
x
f
Z
5
Intrinsic Camera Parameters

• Determine the intrinsic parameters of a camera
  (with lens)
• What are Intrinsic Parameters?
• (can you name 7?)

6
Intrinsic Parameters
O
X
f
Z
7
Intrinsic Camera Parameters

• Intrinsic Parameters
• Focal Length f
• Pixel size sx , sy
• Image center ox , oy
• (Nonlinear radial distortion coefficients k1 ,
  k2)
• Calibration Determine the intrinsic parameters
  of a camera

8
Why Intrinsic Parameters Matter
9
Questions

• Can we determine the intrinsic parameters by
  exposing the camera to many known objects?
• If so,
• How often do we have to see the object?
• How many features on the object do we need?

10
Example Calibration Pattern
Calibration Pattern Object with features of
known size/geometry
11
Harris Corner Detector(see slides in last
lecture)
12
Intrinsics and Extrinsics

• Intrinsics
• Focal Length f
• Pixel size sx , sy
• Image center ox , oy
• Extrinsics
• Location and orientation of k-th calib. pattern

13
Calibration

• Known calibration object, many views
• Compute intrinsics and extrinsics
• (Retain intrinsics, toss extrinsics)

14
Why Tilt the Board?
15
Experiment 1 Parallel Board
16
Projective Perspective of Parallel Board
30cm
10cm
20cm
17
Experiment 2 Tilted Board
18
Projective Perspective of Tilted Board
30cm
10cm
20cm
500cm
50cm
100cm
19
Perspective Camera Model

• Step 1 Transform into camera coordinates
• Step 2 Transform into image coordinates

20
Perspective Camera Model

• Step 1 Transform into camera coordinates
• Step 2 Transform into image coordinates

21
The Full Perspective Camera Model
22
The Calibration Problem

• Given
• Calibration pattern with N corners
• K views of this calibration pattern
• Recover the intrinsic parameters
• Well also recover the extrinsics, but we wont
  care about them

23
Calibration Questions

• Can we determine the intrinsic parameters by
  exposing the camera to many known objects?
• If so,
• How often do we have to see the object?
• How many features on the object do we need?
• Do we need to see object at angle? Yes.

24
Todays Goals

• Calibration Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software

25
Calibration constraints

• Step 1 Transform into camera coordinates
• Step 2 Transform into image coordinates

26
Camera Calibration
27
Calibration by nonlinear Least Squares

• Least Mean Square
• Gradient descent

28
The Calibration Problem Quiz

• Given
• Calibration pattern with N corners
• K views of this calibration pattern
• How large would N and K have to be?
• Can we recover all intrinsic parameters?

29
Intrinsic Parameters, Degeneracy
O
X
f
Z
30
Summary Parameters, Revisited

• Extrinsic
• Rotation
• Translation
• Intrinsic
• Focal length
• Pixel size
• Image center coordinates

31
The Calibration Problem Quiz

• Given
• Calibration pattern with N corners
• K views of this calibration pattern
• How large would N and K have to be?
• Can we recover all intrinsic parameters?

NO
32
Constraints

• N points
• K images ? 2NK constraints
• 4 intrinsics (distortion 2)
• 6K extrinsics
• ? need 2NK 6K4
• ? (N-3)K 2

Hint may not be co-linear
33
The Calibration Problem Quiz
need (N-3)K 2
Hint may not be co-linear
34
Problem with Least Squares

• Many parameters (slow)
• Many local minima! (slower)

35
Todays Goals

• Calibration Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software

36
SVD Solution

• Replace rotation matrix by arbitrary matrix
• Transform into linear set of equations
• Solve via SVD
• Enforce rotation matrix
• Solve for remaining parameters

37
Perspective Camera Model

• Step 1 Transform into camera coordinates
• Step 2 Transform into image coordinates

38
Homogeneous Coordinates
(Homogeneous Coordinates)
(nonlinear perspective projection)
39
Affine Problem Relaxation
?
40
Affine Problem Relaxation
?
41
Calibration via SVD see Trucco/Verri
42
Calibration via SVD
Ngt7 points, not coplanar
43
Calibration via SVD
44
Calibration via SVD
Known X Y Z, x, y Unknown v
A has rank 7 (without proof)
45
Calibration via SVD

• Remaining Problem
• See book

46
Summary, SVD Solution

• Replace rotation matrix by arbitrary matrix
• Transform into linear set of equations
• Solve via SVD
• Enforce rotation matrix (see book)
• Solve for remaining parameters (see book)

47
Comparison

• Nonlinear least squares
• Gaussian image noise
• Many local minima
• Iterative
• Can incorporate non-linear distortion

• Singular Value Decomp.
• Gaussian parameter noise (algebraic)
• No local minima
• Closed form
• Cannot handle distortion

48
Todays Goals

• Calibration Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software

49
Homogeneous Coordinates

• Idea In homogeneous coordinates most operations
  become linear!
• Extract Image Coordinates by Z-normalization

50
Todays Goals

• Calibration Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software

51
Advanced CalibrationNonlinear Distortions

• Barrel and Pincushion
• Tangential

52
Barrel and Pincushion Distortion
tele
wideangle
53
Models of Radial Distortion
distance from center
54
Image Rectification (to be continued)
55
Distorted Camera Calibration

• Set k1k20, solve for undistorted case
• Find optimal k1,k2via nonlinear least squares
• Iterate
• ?Tends to generate good calibrations

56
Tangential Distortion
cheap CMOS chip
cheap lens
image
cheap glue
cheap camera
57
Todays Goals

• Calibration Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software

58
Calibration Software Matlab
59
Calibration Software OpenCV
60
State-of-the-art calibration

• Z. Zhang Flexible Camera Calibration By Viewing
  a Plane From Unknown Orientations (1999)
• Solves correspondence problem
• Works with planar calibration pad
• Works well in practice

61
Your Homework Assignment
62
Summary

• Calibration Problem definition
• Solution by nonlinear Least Squares
• Solution via Singular Value Decomposition
• Homogeneous Coordinates
• Distortion
• Calibration Software
• NOT DISCUSSED correspondence