Applying Photogrammetric Bundle Adjustment To Nonlinear Least Squares Optimization Problems

1
Applying Photogrammetric Bundle Adjustment To
Nonlinear Least Squares Optimization Problems

• David Schug
• AMSC 662
• 11/30/2004

2
Theory and Problem Formation
Setting Up Relevant Coordinate Systems

• Place targets distributed evenly over the surface
  of all the objects that will be measured
  reference object and target object
• Survey objects and reduce data points (X, Y, Z)
  so origins are relative to a particular point on
  each object nose (for reference) and center of
  gravity (for target)
• Objects are stationed in an initial state and a
  tie-in survey is performed to get position of
  target relative to reference object

3
Theory and Problem Formation
Capturing Images and Tracking

• Set up cameras on the boundaries along the x and
  y axis of the reference object and survey their
  positions relative to the origin of the reference
  object
• Cameras record images at a particular frame rate
  image per .005 seconds
• Targets on the reference object and the target
  object are tracked and recorded as (u,v)
  coordinate positions in the u-v image plane

4
Theory and Problem Formation
Correcting Lens Distortion  

• Distortion must be corrected by determining the
  mapping that will adjust each measured u, v
  target position to a corrected u, v position for
  each target point in the image plane
• A nonlinear least squares optimization problem
  must be solved to find the u0v0alphaxyzhprf that
  will minimize the error between the corrected u,
  v and the ideal u, v
• Objective function must be constructed using
  projective geometry and radial correction.
  Optimizer lssqnonlin will apply the objective
  function iteratively until the global minimum is
  reached. This global minimum will be applied to
  each measured u, v to get the correct (u, v)
  for each measured (u, v).

5
Theory and Problem Formation
Orienting the Camera

• Determine a camera's position and orientation
  relative to a reference object given an initial
  guess over a sequence of frames
• A second nonlinear least squares optimization
  problem must be solved to find the each cameras
  position (x,y,z) and orientation (h,p,r) over all
  frames
• Objective function must be constructed using
  rotation sequences and projective geometry to
  compute a vector representing the difference
  between calculated and measured image locations
  of the targets. Optimizer lssqnonlin will apply
  the objective function iteratively until the
  global minimum is reached. This global minimum
  represents the position and orientation of the
  camera.
•  

6
Theory and Problem Formation
Two Approaches to Obtaining the 6 degree of
freedom solution

• Solve for each of 3 cameras position
  independently followed by a final solution over
  all the images
• Solve for the 3 cameras positions and the target
  object simultaneously over all the images for a
  jointly global optimal solution

7
Theory and Problem Formation
Computing the Six Degree of Freedom Solution

• Compute the 6 degree of freedom position and
  orientation solution of a target object relative
  to a given reference object over a sequence of
  images.
• Multiple nonlinear least squares optimization
  problems must be solved to find the target
  objects position and orientation relative to the
  reference object given previously solved camera
  positions.
• Objective function must be constructed using
  rotation sequences and projective geometry to
  compute a vector representing the difference
  between calculated and measured image locations
  of the targets. Optimizer lssqnonlin will apply
  the objective function iteratively until the
  global minimum is reached. This global minimum
  represents the position and orientation of the
  target object relative to the reference object.

8
Theory and Problem Formation
Computing the Six Degree of Freedom Solution with
Bundling

• Compute the 6 degree of freedom position and
  orientation solution of 3 cameras and a target
  object relative to a given reference object over
  a sequence of images.
• Multiple nonlinear least squares optimization
  problems must be solved to find the 3 cameras
  and target objects position and orientation
  relative to the reference object for each image.
• Objective function must be constructed using
  rotation sequences and projective geometry to
  compute a large bundled vector representing the
  difference between calculated and measured image
  locations of the targets. Optimizer lssqnonlin
  will apply the objective function iteratively
  until the global minimum is reached. This global
  minimum represents the position and orientation
  of the 3 cameras and target object relative to
  the reference object.

9
Results and Analysis Camera 14
10
Results and Analysis Camera 18
11
Results and Analysis Camera 84
12
Results and Analysis Target Object
13
Costs
14
Conclusions

• Localized Bundle Code is more expensive to
  evaluate
• Camera positions are less reliable but target
  object positions have lower residuals
• This could imply that bundling could solve more
  difficult cases especially with orientation
  parameters of the target object