Andrew C' Gallagher1

1
Using Vanishing Points toCorrect Camera
Rotation
Andrew C. Gallagher Eastman Kodak
Company andrew.gallagher_at_kodak.com
2
Problem

• An unintentionally tilted camera can negatively
  affect image appearance.
• Caused by lightweight cameras that are
  difficult to hold level.
• People prefer imageswhere the horizon is level.
  
• Human can see as little at 1o tilt.

3
Solution

• Vanishing point location can be used to detect
  and correct image tilt resulting from camera
  rotation.
• A vanishing point is the image of a world line at
  infinity.
• Vanishing point location is useful for
• computing focal length Kanatani
• finding principal point Caprile et al.
• determining camera parameters and rotation
  matrixCipolla et al.

4
Vanishing Points

• Parallel scene lines meet at a vanishing point in
  the image.

Vertical Line Vanishing Point
Horizontal Line Vanishing Point
5
The Camera Model

• The camera model describes the projection of 3D
  world to 2D camera plane.
• K is a 3x3 matrix of the internal camera
  parameters.
• R is a 3x3 matrix describing the rotation from
  the world to the camera frame.
• T is a 3x1 matrix describingtranslation between
  the world and camera coordinate frame.
• Assume no skew, square pixels. The vanishing
  points of world directions are

world coordinate frame
camera coordinate frame
6
The Rotation Matrix

• R is any matrix in the special orthogonal group
  SO(3).
• In practice the camera positions used by typical
  consumers follow a fairly predictable nonlinear
  distribution.
• This distribution is then used to find where
  vanishing points will occur.

7
Camera Position Analysis
World rotation by q about the Y-axis
Default Position

World rotation by f about the X-axis
World rotation about the Z-axisTILTED IMAGE
8
Camera Position Analysis

• This position model encompasses all preferred
  camera positions.
• The vanishing point associated with vertical
  world direction (Y-axis) is constrained to fall
  on the image y-axis.
• The horizon is parallel to image x-axis.

Rotation about both X- and Y- axes

Location of Vy
Location of Vx or Vz
9
Camera Position Analysis

• The original rotation matrix is multiplied by a
  rotation about the Z-axis.
• The new vanishing points are simply rotated by
  the same amount!
• In essence, the rotation of the camera from the
  ideal position is equivalent to the rotation of
  the vanishing points.

Additional rotation about the Z-axis

Location of Vy
Location of Vx or Vz
10
Ground Truth Analysis

• 357 vanishing points were manually labeled to
  compare with expected distribution.
• 160 vertical (Vy) vanishing points197 horizontal
  (Vx or Vz) vanishing points.
• The match is visually good.

Location of Vy
Location of Vx or Vz
EXPECTED DISTRIBUTION
MEASURED DISTRIBUTION
11
Vanishing Point Classification

• The vertical and horizontal vanishing point
  distributions are well-separated.
• A classifier can be used to identify vertical
  vanishing points.
• The camera rotation is found from the vertical
  vanishing point.
• On ground truth, only 2vanishing points
  (0.6)are misclassified.

Vertical vanishing point classifier
12
The Tilt Correction Algorithm

• Find vanishing points
• Identify vertical vanishing points
• Compute camera rotation angle b from the vertical
  vanishing point
• Compute correction angle bc according to table
• Rotate image
• The rotated image can beshown to be equivalent
  to capturing with a camerahaving no
  componentof rotation about the Z-axis.

13
Vanishing Point Detection

• Initial work by Barnard 1983.
• Line Segment Detection
• Lines are found by calculating local gradients,
  then clustering, or by using Hough transform.
• Line Intersection Computation
• Intersections of the lines are found. Line
  intersections are possible locations of a
  vanishing point.
• Maximum Detection
• A detected vanishing point is hypothesized to be
  at a location of many line intersections.

14
Vanishing Point Detection
Original Image
Lines associated with 1st VP.
Plot of all line segment Intersections (Higher
probabilities are red).
Detected Vanishing Points
Lines associated with 2nd VP.
Detected Line Segments
15
Algorithm Results
Lines associated with vertical VP
Original
Corrected
16
Algorithm Results
Lines associated with vertical VP
Original
Corrected
17
Conclusions

• Rotation of the camera about the principal axis
  moves the vertical vanishing point from the image
  y-axis.
• This novel algorithm corrects a tilted image by
  detecting the vertical vanishing point, and
  determining the magnitude of camera rotation.