Multiple View Geometry

1
Multiple View Geometry
2
THE GEOMETRY OF MULTIPLE VIEWS

• Epipolar Geometry
• The Essential Matrix
• The Fundamental Matrix
• The Trifocal Tensor
• The Quadrifocal Tensor

Reading Chapter 10.
3
Epipolar Geometry

• Epipolar Plane

• Baseline

• Epipoles

• Epipolar Lines

4
Epipolar Constraint

• Potential matches for p have to lie on the
  corresponding
• epipolar line l.

• Potential matches for p have to lie on the
  corresponding
• epipolar line l.

5
Epipolar Constraint Calibrated Case
Essential Matrix (Longuet-Higgins, 1981)
6
Properties of the Essential Matrix
T

• E p is the epipolar line associated with p.
• ETp is the epipolar line associated with p.
• E e0 and ETe0.
• E is singular.
• E has two equal non-zero singular values
• (Huang and Faugeras, 1989).

T
7
Epipolar Constraint Small Motions
To First-Order
Pure translation Focus of Expansion
8
Epipolar Constraint Uncalibrated Case
Fundamental Matrix (Faugeras and Luong, 1992)
9
Properties of the Fundamental Matrix

• F p is the epipolar line associated with p.
• FT p is the epipolar line associated with p.
• F e0 and FT e0.
• F is singular.

T
T
10
The Eight-Point Algorithm (Longuet-Higgins, 1981)
11
Non-Linear Least-Squares Approach (Luong et al.,
1993)
Minimize
with respect to the coefficients of F , using an
appropriate rank-2 parameterization.
12
Problem with eight-point algorithm
linear least-squares unit norm vector F
yielding smallest residual What happens when
there is noise?
13
The Normalized Eight-Point Algorithm (Hartley,
1995)

• Center the image data at the origin, and scale
  it so the
• mean squared distance between the origin and the
  data
• points is sqrt(2) pixels q T p , q T
  p.
• Use the eight-point algorithm to compute F from
  the
• points q and q .
• Enforce the rank-2 constraint.
• Output T F T.

i
i
i
i
i
i
T
14
Weak-calibration Experiments
15
Epipolar geometry example
16
Example converging cameras
courtesy of Andrew Zisserman
17
Trinocular Epipolar Constraints
These constraints are not independent!
18
Trinocular Epipolar Constraints Transfer
Given p and p , p can be computed as the
solution of linear equations.
1
2
3
19
Trinocular Epipolar Constraints Transfer

• problem for epipolar transfer in trifocal plane!

There must be more to trifocal geometry
image from Hartley and Zisserman
20
Trifocal Constraints
21
Trifocal Constraints
Calibrated Case
All 3x3 minors must be zero!
Trifocal Tensor
22
Trifocal Constraints
Uncalibrated Case
Trifocal Tensor
23
Trifocal Constraints 3 Points
Pick any two lines l and l through p and p .
2
3
2
3
Do it again.
24
Properties of the Trifocal Tensor
T
i

• For any matching epipolar lines, l G l
  0.
• The matrices G are singular.
• They satisfy 8 independent constraints in the
• uncalibrated case (Faugeras and Mourrain, 1995).

2
1
3
i
1
Estimating the Trifocal Tensor

• Ignore the non-linear constraints and use linear
  least-squares
• Impose the constraints a posteriori.

25
T
i
For any matching epipolar lines, l G l
0.
2
1
3
The backprojections of the two lines do not
define a line!
26
Trifocal Tensor Example
108 putative matches
18 outliers
(26 samples)
95 final inliers
88 inliers
(0.19)
(0.43) (0.23)
courtesy of Andrew Zisserman
27
Trifocal Tensor Example
additional line matches
images courtesy of Andrew Zisserman
28
Transfer trifocal transfer
(using tensor notation)
doesnt work if lepipolar line
image courtesy of Hartley and Zisserman
29
Image warping using T(1,2,N)
(Avidan and Shashua 97)
30
Multiple Views (Faugeras and Mourrain, 1995)
31
Two Views
Epipolar Constraint
32
Three Views
Trifocal Constraint
33
Four Views
Quadrifocal Constraint (Triggs, 1995)
34
Geometrically, the four rays must intersect in P..
35
Quadrifocal Tensor and Lines
36
Quadrifocal tensor

• determinant is multilinear
• thus linear in coefficients of lines
  !
• There must exist a tensor with 81 coefficients
  containing all possible combination of x,y,w
  coefficients for all 4 images the quadrifocal
  tensor

37
Scale-Restraint Condition from Photogrammetry