Estimation of Epipolar Geometry and 3D Reconstruction from Uncalibrated Image Pairs

1
Estimation of Epipolar Geometry and 3D
Reconstruction from Uncalibrated Image Pairs

• Michael Unger
• Treffen (2008) des ITG FA 3.2

2
Outline

• Motivation
• Fundamentals
• Outlier Removal Probabilistic Epipolar Distance
• Lens Distortion
• Point Correspondence Analysis
• Results
• Motion compensated background subtraction

3
Motivation

• 3D Reconstruction from an uncalibrated image
  pair

Reconstruction and calibration requires many and
accurate point correspondences
4
Triangulation
Calculation of a 3D space point by intersecting
rays of two views

• Required information
• Set of corresponding image points and
• Focal length, camera origin ? K
• Rotation R, translation t
• Lens distortion parameter
• Drawback required information is noisy or even
  not known

5
Epipolar Geometry
X
epipolar line of x
Mapping of a ray through the first camera into
the second camera
l
xx,yT
x
e
e
C
C

• The fundamentalmatrix F is a 3x3 homogeneous
  matrix, algebraically encapsulating the epipolar
  geometry

6
Properties of the Fundamental Matrix F

• Epipolar lines
• F is scale invariant and satisfies
• ? F has seven degrees of freedom
• Calculation of F by solving at least seven
  equations of the form
• Connection between F and the five external camera
  parameters
• Internal camera parameters are linked to F via
  two independent Kruppa equations.
• Drawback vulnerable to noise ? undistorted
  point correspondences needed

7
Epipolar Geometry Estimation

• Standard methods
• (Normalized) 8-Point algorithm
• RANSAC-algorithm

• Drawback Distorted epipolar line due to
• Noise
• Outlier

? Outlier removal / detection of more point
correspondences necessary
Improvement Usage of probabilistic epipolar
distance instead of Euclidean distance
8
Outlier Removal using a Probabilistic Epipolar
Distance

• Compute the covarianz matrix of F using a
  Monte-Carlo-Simulation
• Calculate the Jacobian of the mapping
• Derive the covarianz matrix of an epipolar line
  by
• Calculate for an image point the mahalanobis
  distance by finding a line going
  through the point and minimizing the equation
• For points sharing the same a conic may
  be computed by

9
Probabilistic Epipolar Distance
Resulting hyperbola for points sharing the same
values (white 0)
10
Outlier Removal
Outliers are visible on left cube
Outliers on the left cube are eliminated.
11
Lens Distortion

• Observation Displacemant of the epipolar lines

k
Drawback Robust point correspondences necessary
12
Point Correspondence Analysis

• Standard methods
• Scale invariant feature transform (SIFT)
• Lucas Kanade Tracker
• Drawbacks
• Sparse number of point correspondences especially
  in homogenous regions
• Feature points neccessary (Lucas Kanade)
• No usage of epipolar constraint ? Outlier
• Chicken/Egg Problem
• A good Epipolar Geometry requires accurate p.c.
• P.c. analysis benefits from an accurate epipolar
  geometry

13
Homographies

• has eight degrees of freedom
• Using only three degrees of freedom are
  necessary to derive any homography

e
e
C
C

• Each point correspondence provide one
  information
• Three 3D space points define a plane
• ? Three point correspondences define a
  homography

14
Enhanced Point Correspondence Estimation
Epipolar line
Triangulation in image A
Triangulation in image B

• Iterative algorithm
• Delaunay triangulation of feature points
• Calculation of F from point correspondences
• Feature point detection within triangles
• Feature point prediction using H
• Matching
• Triangulation update

15
Accuracy of Epipolar Geometry Estimation

• Improvement of the epipolar geometry estimation
  while detecting additional point correspondences
  (Room images)

Absolute deviation of the epipolar geometry in
pix
Iteration steps (higher of correspondences)
16
Results 3D Reconstruction

• 3D Reconstruction from an uncalibrated image pair

17
Reconstruction
18
Motion Compensated Background Subtraction
camera models
camera models
current frame


Original
Background
Result
19
Motion Compensated Background Subtraction

• Camera model estimation
• Generation of an artificial background
• Background subtraction

20

• Thank you for your attention!