Igert Computer Vision Lab

1
Igert Computer Vision Lab

• Adapted from Serge Belongie and Josh Wills
• TAs Adam Bickett, Paul Ruvolo,
• Cynthia Taylor, Matt Tong

• Corner Detection
• Feature tracking
• Recovery of 3D structure from motion

2
Corner Detection

• Sharp change in the angle of the gradient
• Corner is point of intersection of the tangents

3
Corner Detection

• In real images, corners will look more like this.
  
• Estimate point of intersection of the tangents

4
Corner Detection
Want the point x0 with min perpendicular distance
to all the tangent lines through the points x in
the 5x5 neighborhood.
Dx(x) describes the distance from x to the
tangent line lx multiplied by the gradient
magnitude.
5
Least squares solution
Solve by taking derivative and setting 0.
6
Least Squares Solution
X0 contains subpixel coordinates of the corner
X(1) X(2)
b
7
Least Squares Solution(another way to think
about it)

• ?I(x)Tx ?I(x)Tx

M x y MTM x MTy
Linear system of equations over all x in 5x5
neighborhood. Solve using standard regression
x
A b
8
A-1 only exists here
Size of the 2nd eigenvalue of A measures
cornerness
9
(No Transcript)
10
Feature Matching / Tracking

• Greedy Nearest Neighbor Feature Matching
• Optic Flow (Horn Schunk)
• Lucas-Kanade solution to the aperture problem

11
Greedy Nearest Neighbor Feature Matching

• Run interest point operator (e.g. Corner
  detector) on image pair.
• Compute distance matrix containing distances
  between each corner detected in image 1 and image
  2.
• Find closest corner Cj in image 2 to corner 1 in
  image 1. Remove Cj from future comparisons and
  repeat for corner 2 of image 1.

12
Optic Flow

• We explain I2 fully just by moving the points in
  I1 around. No brightness added or lost
  (Brightness constraint).

• We get rid of the subscript and define the
  brightness I as a function of time t and the
  point positions x(t).

13
Optic Flow

• We get a first approximation to
• using the Taylor series expansion. i.e.
  start with I(t) and move in the direction of the
  first derivative.

• Still using brightness constraint ( I(x(t),t)
  RHS ) the fist term cancels

14
Optic Flow

• Dividing by

Velocity in x direction
Velocity in y direction
Spatial Derivative x direction
Temporal derivative
Spatial Derivative y direction
15

• One equation for two unknowns

16
Aperture problem
Want to estimate motion in neighborhoods
containing corners
17
Aperture Problem Demo

• http//psylux.psych.tu-dresden.de/i1/kaw/diverses
  20Material/www.illusionworks.com/html/barber_pole.
  html

18
Lucas-Kanade Neighborhood Solution
We now have NxN equations for 2 unknowns. Solve
using standard least-squares regression.
19
Standard least-squares regression
u pinv(A)b
20
Note
ATA is the second-moment matrix seen in the
corner detector. As with corner detection the
solution is unique only when ATA is rank 2. i.e.
only where there are corners in the neighborhood.
21
Recovering 3D structure from matched points
(Structure From Motion)
22
Orthographic camera model
23
The General Problem

• 3 frames of at least 4 points, correspondences
  known.

(xij,yij) are known. Position of point j in frame
i. Want T Translation of the image origin
from the world origin R Rotation of the image
axes S Scale of the image axes
24
Translate to object centered coordinates
Center the coordinates in each frame
Now all the coordinates are wrt the centroid of
the object
25
Factorize W RS
i1 --- i2 --- j1 --- j2 ---
Define
(2N x 3)
We know W and want to find RS. Factorize W into
RS using SVD.
26
Factorize W RS

• Apply SVD and take the first 3 columns

i1 --- i2 --- j1 --- j2 ---
The columns of R are orthogonal, but want rows to
contain the image axes ii ji.
27
Enforce Constraints on R
?
?
Find Q such that
Then
(Solve for Q using methods for quadratic system
of equations)