Mixing Catadioptric and Perspective Cameras

1
Mixing Catadioptric and Perspective Cameras
Peter Sturm INRIA Rhône-Alpes France
2
Introduction
3
Introduction
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Introduction
Existing results
- epipolar geometry between omnidirectional
cameras
- motion estimation
- (self-) calibration
- ...
9
Our Goals
Study the geometry of hybrid stereo systems
(omnidirectional and perspective cameras)
- epipolar geometry
- trifocal tensors
- plane homographies
Applications
- motion estimation
- calibration
- self-calibration, calibration transfer
- 3D reconstruction
- ...
10
Plan
? Camera models
Epipolar geometry of hybrid systems
Derivation of matching tensors
Applications
Conclusions
11
Camera Models
Perspective and affine cameras
12
Camera Models
Central catadioptric cameras
mirror (surface of revolution of a conic)
camera
virtual optical center
13
Camera Models
Central catadioptric cameras
mirror (surface of revolution of a conic)
camera
virtual optical center
calibration
14
Camera Models
Types of central catadioptric cameras
hyperbola perspective camera
parabola affine camera
...
15
Plan
Camera models
? Epipolar geometry of hybrid systems
Derivation of matching tensors
Applications
Conclusions
16
Epipolar Geometry
epipolar plane
epipolar line
epipolar line
q
q
epipole
17
Epipolar Geometry
epipolar conic
18
Epipolar Geometry
epipolar line
epipole
epipolar conic
epipole
epipoles
19
Epipolar Geometry Example
20
Epipolar Geometry
epipolar line
epipole
epipolar conic
epipoles
purely perspective case
21
Epipolar Geometry
epipolar line
epipole
epipolar conic
epipoles
22
Epipolar Geometry Example
23
Epipolar Geometry
BUT...
Until now, linear epipolar relation only found
for
any combination of perspective, affine or
para-catadioptric cameras (parabolic mirrors)
Not yet for
other omnidirectional cameras than
para-catadioptric ones (e.g. based on
hyperbolic mirrors)
24
Epipolar Geometry
Special case
combination of perspective and
para-catadioptric cameras
epipolar conics are circles
F is of dimension 4x3
the lifted coordinates are
25
Epipolar Geometry
Epipoles
is of rank 2
The epipole of the perspective camera is the
right null-vector of F
F has a one-dimensional left null-space ?
the two epipoles of the catadioptric camera are
the left null-vectors that are valid
lifted coordinates (quadratic
constraint)
26
Plan
Camera models
Epipolar geometry of hybrid systems
? Derivation of matching tensors
Applications
Conclusions
27
Matching Tensors
Multi-linear relations between coordinates of
correponding image points
Purely perspective case derivation based on
linear equations representing projections
(3D ? 2D)
Here equations for back-projection (2D ? 3D
directions)
- perspective cameras
28
Matching Tensors
Linear equations representing back-projections
- perspective cameras
- para-catadioptric cameras
29
Matching Tensors
Putting the equations together
30
Matching Tensors
Putting the equations together
31
Matching Tensors
Straightforward extension to more than 2
views ...
32
Plan
Camera models
Epipolar geometry of hybrid systems
Derivation of matching tensors
? Applications
- Self-calibration of omnidirectional cameras
from fundamental matrices
- Calibration transfer from an omnidirectional
to a perspective camera
- Self-calibration of omnidirectional cameras
from a plane homography
Conclusions
33
Applications
Self-calibration of omnidirectional cameras
from fundamental matrices
para-catadioptric camera has 3 intrinsic
parameters
representation as a 4-vector of homogeneous
coordinates
this vector is in the left null-space of the
fundamental matrix of this camera, defined
with respect to any other camera
(perspective, affine, catadioptric)
? self-calibration is possible from two or more
fundamental matrices
34
Applications
Calibration transfer from an omnidirectional to
a perspective camera
input - calibration of a
para-catadioptric camera - fundamental
matrix with a perspective camera
? closed-form solution for the focal length of
the perspective camera
35
Applications
Self-calibration of omnidirectional cameras
from a plane homography
input - plane homography H with a
perspective camera
? recovery of intrinsic parameters of
para-catadioptric camera (given by
null-vector of H)
36
Plan
Camera models
Epipolar geometry of hybrid systems
Derivation of matching tensors
Applications
? Conclusions
37
Conclusions
Multi-linear matching relations between
perspective, affine and para-catadioptric cameras
Applications in calibration,
self-calibration, motion estimation, 3D
reconstruction, ...
Open questions
Fundamental matrix etc. for hyper-catadioptric
cameras ?
Plane homographies for the inverse
direction ?
Perspectives
Hybrid trifocal tensors for line images
Multi-view 3D reconstruction for hybrid systems
38
Mixing Catadioptric and Perspective Cameras
Peter Sturm INRIA Rhône-Alpes France