Recognition and 3D Reconstruction from Video

1
Recognition and 3D Reconstruction from Video
David Nistér
2
50 Thousand Images
3
110,000,000Images in5.8 Seconds
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
Scalable Recognition with a Vocabulary Tree
David Nistér, Henrik Stewénius
8
Towards Urban 3D Reconstruction From Video
A. Akbarzadeh, J.-M. Frahm, P. Mordohai, B.
Clipp, C. Engels, D. Gallup, P. Merrell, M.
Phelps, S. Sinha, B. Talton, L. Wang, Q. Yáng, H.
Stewénius, R. Yang, G. Welch, H. Towles, D.
Nistér and M. Pollefeys
9
(No Transcript)
10
(No Transcript)
11
MP
12
Video collection
2x4 cameras 1024x768_at_30Hz
13
(No Transcript)
14
Video Data
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Outline

• Feature Extraction and Description
• Matching, Tracking and Indexing
• Geometry
• Surface Reconstruction

19
The transformation hierarchy
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
Structure and Motion
Feature Matching
Feature Detection
Original Video
3D Reconstruction
26

• Rotation

• Viewpoint Change

• Lighting Variation

• Scale Change

27
Invariance or Covariance

• Detection and image transformation commutes

Detect (Transform(I))Transform(Detect(I))
28
Rotation-Invariant Detection

• Moravec
• Förstner
• Harris

29
Feature Detection
Harris Corners
Structure and Motion
Feature Matching
Feature Detection
Original Video
3D Reconstruction
30
Feature Detection
Harris Corners
31
Feature Detection
Harris Corners
32
Feature Detection
Harris Corners
Second Moment Matrix
33
Feature Detection
5x5 Max
Image
k
Saturation
Features
34
Feature Detection
35
RotationScale Invariant Detection

• DoG Points
• Lindeberg, Schmid Mohr, Lowe

36
DoG Points

• Blob detector

-
Video In
37
Affine Invariant Regions

• Tuytelaars Van Gool
• Mikolajczyk and Schmid
• Matas et al.

38
Harris and Hessian Affine

• Mikolajczyk and Schmid

39
MSER

• Matas et al.
• Similar to watershed, but thresholded at minimal
  change rather than segmented when watersheds join

40
MSER

• Extremal regions are continuous-invariant
• MSERs are affine invariant if growth is measured
  in relative terms

41
Demonstration of live feature tracking and MSERs
42
Vertical Resampler
Gradient Orientation Histogram
SIFT Descriptor
Circular regions
Oriented Regions
Upright Elliptical Regions
Frame-to-Frame Tracking
Region Resampler
Elliptical Regions
Tracks of Affine Invariant Regions and
Corresponding Descriptors
MSER
Video In
43
Selecting a coordinate system
44
Region Description

• Image Patch
• Normalized Image Patch
• SIFT Descriptor
• DCT Descriptors
• Wavelets

45
SIFT Descriptor
46
Structure and Motion
Feature Matching
Feature Detection
Original Video
3D Reconstruction
47
2D Tracking
KLT
Harris
HC
48
Feature Matching/Tracking
Normalized Correlation
49
Feature Matching/Tracking
Only retain bidirectional matches No loops
because of symmetry d(a,b)d(b,a)
50
Feature Matching/Tracking
Structure and Motion
Feature Matching
Feature Detection
Original Video
3D Reconstruction
51
Feature Matching/Tracking
52
Feature Matching/Tracking
53
Feature Matching/Tracking
54
Matching vs Tracking

• Detection, while a tremendous strength in terms
  of scalability, is a weakness for repeatability

55
KLT Tracker
Harris Tracker
56
GPU KLT
work of Sudipta Sinha
Image 1024 x 768 1000 features
ms
57
GPU-KLT
58
Indexing

• Fighting the curse of dimensionality
• Locality Sensitive Hashing (LSH)
• K-d tree
• Vocabulary Tree

Find nearest neighbor
59
tf-idf

• Term Frequency Inverse Document Frequency
• Is a weighting of words in a document

(n/N) log (D/d)
60
Clustering

• K-Means
• K-Medioids
• Mean-Shift
• Spectral Clustering
• Graph-Cuts

61
K-means
62
Mean-Shift
63
Spectral-Clustering
Break into eigen-modes
64
Graph-Cuts
65
Machine Learning

• When parametric invariance is insufficient
• Supervised,Unsupervised,Semisupervised
• Support Vector Machines (SVMs)
• Boosting
• Neural Nets

66
Scalability
If we can get repeatable, discriminative
features, then recognition can scale to very
large databases using the vocabulary tree and
indexing approach described in Nistér
Stewénius CVPR 2006.
67
Vocabulary Tree
68
Vocabulary Tree
69
Vocabulary Tree
70
(No Transcript)
71
Adding, Querying and Removing Images at full speed
Query
Add
Remove
72
Training and Addition are Separate
Common Approach
Our approach
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
(No Transcript)
80
(No Transcript)
81
(No Transcript)
82
(No Transcript)
83
(No Transcript)
84
(No Transcript)
85
(No Transcript)
86
(No Transcript)
87
(No Transcript)
88
(No Transcript)
89
(No Transcript)
90
(No Transcript)
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94
(No Transcript)
95
(No Transcript)
96
Performance
97
(No Transcript)
98
Size Matters
99
(No Transcript)
100
(No Transcript)
101
Geometric Verification
102
Robust to Clutter and Occlusion

• Local Regions
• Like Web-search

103
Geometry

• Demonstration of real-time camera tracking

104
(No Transcript)
105
Visual Odometrywork with Oleg Naroditsky and Jim
Bergen
106
Visual Odometrywork with Oleg Naroditsky and Jim
Bergen

• 365 m without loss of tracking
• 350 m ( 3.5 minutes) without GPS
• Error in distance traveled 1
• Accumulated error in position 3-5
• e.g. 10m over 350m

North
East
107
Visual Odometrywork with Oleg Naroditsky and Jim
Bergen
108
Visual Odometrywork with Oleg Naroditsky and Jim
Bergen
109
3D Tracker
110

• Large scale model produced purely from video (no
  GPS/INS)

111
(No Transcript)
112
(No Transcript)
113

• Large scale model produced purely from video (no
  GPS/INS)

114
Geo Registered Cameras(With INS Data)
115
GPS Data Gathering

• Garmin GPS16
• 200 unit
• 1Hz updates
• Records
• Latitude-Longitude
• Pseudo-range up to 12 satellites
• Satellites clock

116
3D Tracking and Geo-registration
117
3D Tracking and Geo-registration
118
Lever arm calibration

• lever arm from drawings

refined lever arm
119
Lever arm calibration
120
(No Transcript)
121
Geometry Tools
122
Bundle Adjustment
123
Trust Region Methods
x
124
Trust Region Methods
x
dx
125
Trust Region Methods
x
dx
126
Trust Region Methods
x
dx
127
Least squares with Gauss Newton Approximation
Results in cost function approximation
Identify Gradient
and Hessian Approximation
128
Bundle Adjustment
Block LU factorization
Multiply by
Multiply by
First order sparsity
129
Bundle Adjustment
Block LU factorization
Multiply by
Multiply by
130
Bundle Adjustment
Range
Bundle Adjuster
Domain
131
Reprojection constraint
World point
Image point
3DCamera
Radial Distortion
spaf
RC
RP
T
132
Bundle Adjustment
133
Bundle Adjustment
134
3D Tracking
Bundled
SBET Only
135
Structure and Motion
Feature Matching
Feature Detection
Original Video
3D Reconstruction
136
Estimate or posterior likelihood output
Hypothesis Generator
Probabilistic Formulation
Precise Formulation
Data Input
137
RANSAC- Random Sample Consensus
138
RANSAC- Random Sample Consensus
Line Hypotheses
Points
139
RANSAC
?
Hypotheses
500
Observations
1000
500 x 1000 500.000
140
Preemptive RANSAC
Depth-first Preemption
Hypotheses
500
Observations
1000
500 x ???? ???????
141
Preemptive RANSAC
Breadth-first Preemption
Hypotheses
500
Chunksize
100
Observations
1000
500 x 200 100.000
Overhead 100 microseconds
142
Preemptive RANSAC
Observed Tracks
143
Preemptive RANSAC
144
Preemptive RANSAC
145
Relative Orientation
146
Calibrated vs Uncalibrated
147
Constraints
148
Constraints
SingularValues
149
2 Views
3 Views
6p Quan, 1994
8p von Sanden, 1908 Longuet-Higgins, 1981
4p Nister, Schaffalitzky, 2004
7p R. Sturm, 1869
5p Nister, 2003
6p Philip, 1996
5p Kruppa 1913 Nister 2003
150
(No Transcript)
151
The Epipoles and the Epipolar Line Homography
152
The Epipolar Constraint
h
153
The Kruppa Constraints
h
154
The Five Point Problem
Given five point correspondences,
What is R,t ?
E. Kruppa, Zur Ermittlung eines Objektes aus zwei
Perspektiven mit Innerer Orientierung, 1913.
155
O. Faugeras and S. Maybank, Motion from Point
Matches Multiplicity of Solutions, 1990.
J. Philip, A Non-Iterative Algorithm for
Determining all Essential Matrices Corresponding
to Five Point Pairs, 1996.
B. Triggs, Routines for Relative Pose of Two
Calibrated Cameras from 5 Points, 2000.
D. Nister, An Efficient Solution to the
Five-Point Relative Pose Problem, 2002.
156
The solution is minimal in two respects
157
Nr of Roots
Average 4.55
158
Nr of Solutions
Average 2.74
159
10 Solutions
0.067, 0.287 lt gt 0.329,1.297 0.254,
0.0646 lt gt 0.523,1.0807 0.239, -0.213 lt
gt 0.517,0.645 -0.710, -0.693 lt gt
-0.141,0.157 0.661, -0.307 lt gt 0.950,
0.773
160
The 5-point algorithm (Nistér PAMI 04)
161
The 5-point algorithm (Nistér CVPR 03)
E
R,t
162
The 5-point algorithm (Nistér PAMI 04)
R,t
E
163
The 5-point algorithm (Stewénius et al)
10 x 10 Action Matrix
Eigen-Decomposition
R,t
E
164
5-Point Matlab Executable
Recent Developments on Direct Relative
Orientation, Henrik Stewenius, Christopher
Engels, David Nister, ISPRS Journal of
Photogrammetry and Remote Sensing
www.vis.uky.edu/dnister
165
(No Transcript)
166
Noise
Minimal Cases, Sideways Motion
Depth 0.5 Baseline 0.1 Field of View 45 degrees
167
Direction
50 points
Depth 0.5 Baseline 0.1 Field of View 45 degrees
168
Baseline
Minimal Cases, Sideways Motion
Depth 0.5 Baseline 0.1 Field of View 45 degrees
169
Easy Conditions
Realistic Conditions
Correct Calibration
170
Focal Length Miscalibration
0.7
0.5
0.3
0.05
3.0
2.0
1.5
1.3
171
Planar Ambiguity, Uncalibrated
2Degrees of Freedom
172
Planar Ambiguity, Calibrated
2-Fold or Unique
173
Depth
174
The 3 View 4 Point Problem
175
(No Transcript)
176
(No Transcript)
177
How Hard is this Problem?
178
Approximately This Hard
179
(No Transcript)
180
(No Transcript)
181
(No Transcript)
182
(No Transcript)
183
(No Transcript)
184
(No Transcript)
185
(No Transcript)
186
(No Transcript)
187
Uncertainty in Epipolar Geometry
work with Chris Engels
Single Estimate often ill posed
Representation of posterior likelihood well
posed, but computationally challenging
188
Uncertainty in Epipolar Geometry
work with Chris Engels
Single Estimate often ill posed
Representation of posterior likelihood well
posed, but computationally challenging
189
Epipoloscope
work with Chris Engels
190
Epipoloscope
work with Chris Engels
8 point
5 point
191
Hypothesis Generators

• Partially data-driven methods
• Five-point epipole
• Three-point epipole (uses intrinsic
  calibration)
• Fully data-driven methods
• Eight-point
• Seven-point
• Five-point (uses intrinsic calibration)

192
Results

• Likelihood image using different methods

Five-Point
Eight-Point
Seven-Point
Three-Point epipole
Five-Point epipole
193
Results

• Convergence of the posterior

194
Results

• Estimation of Confidence Interval
• Confidence estimated by probability mass
  contained within certain interval

True epipole
Confidence interval
195
Results

• Comparison of Confidence Intervals

196
Results

• Comparison of Confidence Intervals
• Fully Data-driven

Five-Point 0.935666
Eight-Point 0.277246
Seven-Point 0.395411
197
Results

• Comparison of Confidence Intervals
• Partially Data-driven

Three-Point epipole 0.937596
Five-Point epipole 0.407995
198
Results

• Baseline Selection
• Choose best pair of frames for pose, stereo, etc.

199
Triangulation
200
Triangulation

• 2 Stages Correction Ideal Triangulation

201
Triangulation

• Rays Intersect lt-gt Rays Coplanar

202
Triangulation

• One parameter family Balance the error

203
Triangulation

• One parameter family Balance the error

x
x
e
e
204
Triangulation

• One parameter family Balance the error

205
Triangulation

• One parameter family Balance the error

• Max-Norm -gt Quartic (Closed form, Nistér)

• L2-Norm -gt Sextic (Hartley Sturm)

• Directional Error -gt Quadratic (Oliensis)

206
Optimal 3 View Triangulation work with Henrik
Stewenius and Fred Schaffalitzky
47 Stationary Points
207
Nr of Stationary Points for Triangulations in N
Views
208
Sampson Approximation
Where
is the covariance matrix of detected image
features and
are the incidence function and its Jacobian
and
209
Sampson Approximation
For two views this leads to
For three views, an approximation of the distance
to trifocal incidence can be found by tensor
contractions and Cramers rule in lt1 microsecond
Assuming Cauchy distribution
210
2D-3D Pose
211
The 3-Point Problem
212
The 3-Point Problem
213
(No Transcript)
214
(No Transcript)
215
(No Transcript)
216
(No Transcript)
217
(No Transcript)
218
(No Transcript)
219
(No Transcript)
220
(No Transcript)
221
(No Transcript)
222
(No Transcript)
223
(No Transcript)
224
(No Transcript)
225
(No Transcript)
226
(No Transcript)
227
(No Transcript)
228
(No Transcript)
229
(No Transcript)
230
(No Transcript)
231
(No Transcript)
232
(No Transcript)
233
(No Transcript)
234
(No Transcript)
235
(No Transcript)
236
(No Transcript)
237
(No Transcript)
238
(No Transcript)
239
(No Transcript)
240
(No Transcript)
241
(No Transcript)
242
(No Transcript)
243
(No Transcript)
244
(No Transcript)
245
(No Transcript)
246
(No Transcript)
247
(No Transcript)
248
(No Transcript)
249
(No Transcript)
250
(No Transcript)
251
(No Transcript)
252
(No Transcript)
253
(No Transcript)
254
(No Transcript)
255
(No Transcript)
256
(No Transcript)
257
(No Transcript)
258
(No Transcript)
259
(No Transcript)
260
(No Transcript)
261
(No Transcript)
262
(No Transcript)
263
(No Transcript)
264
(No Transcript)
265
(No Transcript)
266
(No Transcript)
267
(No Transcript)
268
(No Transcript)
269
(No Transcript)
270
(No Transcript)
271
(No Transcript)
272
(No Transcript)
273
(No Transcript)
274
(No Transcript)
275
(No Transcript)
276
(No Transcript)
277
(No Transcript)
278
(No Transcript)
279
(No Transcript)
280
(No Transcript)
281
(No Transcript)
282
(No Transcript)
283
(No Transcript)
284
(No Transcript)
285
(No Transcript)
286
(No Transcript)
287
(No Transcript)
288
(No Transcript)
289
(No Transcript)
290
(No Transcript)
291
(No Transcript)
292
(No Transcript)
293
(No Transcript)
294
(No Transcript)
295
(No Transcript)
296
(No Transcript)
297
(No Transcript)
298
(No Transcript)
299
(No Transcript)
300
(No Transcript)
301
(No Transcript)
302
(No Transcript)
303
(No Transcript)
304
(No Transcript)
305
(No Transcript)
306
(No Transcript)
307
(No Transcript)
308
(No Transcript)
309
(No Transcript)
310
(No Transcript)
311
(No Transcript)
312
(No Transcript)
313
(No Transcript)
314
(No Transcript)
315
(No Transcript)
316
(No Transcript)
317
(No Transcript)
318
(No Transcript)
319
(No Transcript)
320
(No Transcript)
321
(No Transcript)
322
(No Transcript)
323
(No Transcript)
324
(No Transcript)
325
(No Transcript)
326
(No Transcript)
327
(No Transcript)
328
(No Transcript)
329
(No Transcript)
330
(No Transcript)
331
(No Transcript)
332
(No Transcript)
333
(No Transcript)
334
(No Transcript)
335
(No Transcript)
336
(No Transcript)
337
(No Transcript)
338
(No Transcript)
339
(No Transcript)
340
(No Transcript)
341
(No Transcript)
342
(No Transcript)
343
(No Transcript)
344
(No Transcript)
345
(No Transcript)
346
(No Transcript)
347
(No Transcript)
348
(No Transcript)
349
(No Transcript)
350
(No Transcript)
351
(No Transcript)
352
(No Transcript)
353
(No Transcript)
354
(No Transcript)
355
(No Transcript)
356
(No Transcript)
357
(No Transcript)
358
(No Transcript)
359
(No Transcript)
360
(No Transcript)
361
(No Transcript)
362
(No Transcript)
363
(No Transcript)
364
(No Transcript)
365
(No Transcript)
366
(No Transcript)
367
(No Transcript)
368
(No Transcript)
369
(No Transcript)
370
(No Transcript)
371
(No Transcript)
372
(No Transcript)
373
(No Transcript)
374
(No Transcript)
375
(No Transcript)
376
(No Transcript)
377
(No Transcript)
378
(No Transcript)
379
(No Transcript)
380
(No Transcript)
381
(No Transcript)
382
(No Transcript)
383
(No Transcript)
384
Seamlessly into the classical case
385
Moving Stereo Pair
386
Moving Stereo Pair
387
Moving Stereo Pair
388
6-point pose
Linear, stack 5 point constraints, results in
pencil of cameras
Projects world point onto image line
Correct point by perpendicular projection. Add
constraint and solve uniquely
389
Absolute OrientationStitching
B. Horn, Closed-Form Solution of Absolute
Orientation using Unit Quaternions
390
Absolute OrientationStitching
391
Absolute OrientationStitching
One camera overlap
Projective 4 points, Nistér 01 Calibrated 1
point
392
Geometry-Algebra Dualism
Algebraic Geometry

• Hilberts Nullstellensatz

393
Hypothesis Generation
The 5-Point Relative Pose Problem
Unknown Focal Relative Pose
10
15
2048
The Generalized 3-Point Problem
Microphone-Speaker Relative Orientation
8(4)
The 3 View 4-Point Problem
0 (or thousands)
8-38-150-344-??
394
Wolfgang Gröbner (1899-1980)
Bruno Buchberger
RISC Research Institute for Symbolic
Computation Linz, Austria
395
Suggested Literature

• D. Cox, J. Little, D. OShea, Ideals, Varieties,
• and Algorithms, Second Edition, 1996.
• D. Cox, J. Little, D. OShea, Using Algebraic
  Geometry, Springer 1998.
• T. Becker and Weispfennig, Gröbner Bases, A
  Computational Approach to commutative Algebra,
  Springer 1993.

396
Approach
Begin (online)
Begin (offline)
Pose Problem over R
Pose Problem. Port to Zp
Compute Gröbner basis
Compute number of solutions
Elimination Schedule
Compute Action Matrix
Build matrix based Gröbner basis code
Solve Eigenproblem
Port to R
Backsubstitute
End
End
397
Examples of Solved Problems
6-point generalized relative orientation (64
solutions) (Stewenius, Nistér, Oskarsson and
Åström, Omnivis 2005) 6-point relative
orientation with common but unknown focal length
(15 solutions) (Stewenius, Nistér, Schaffalitzky
and Kahl, CVPR 2005)
398
Audio-Grammetry
work with Henrik Stewenius, Jens Hannemann, Kevin
Donahue
399
Microphone-Speaker Location work with Henrik
Stewenius, Jens Hannemann, Kevin Donahue
400
(No Transcript)
401
Sparse
Dense
402
Sparse Reconstruction
Structure and Motion
Feature Matching
Feature Detection
Original Video
Camera Motion
Dense Reconstruction
Window-based stereo
at multiple scales
Bayesian
Surface
Model Out
Median Fusion
framework
triangulation
driven by
of depth maps
and texturing
graph cuts
403
Dense Reconstruction
404
Stereo

• Feature Based Stereo
• Classical Stereo
• Dynamic Programming
• Belief Propagation
• Graph Cuts
• Color Segmentation
• Plane Sweep
• Level Sets

405
Discontinuity Energy
Dissimilarity Energy
406
Multi-View Depth Reconstruction

• Dynamic Programming

Belief Propagation
407
Dynamic Programming
Image Scanline
Depth
408
Belief Propagation
Image Columns
Image Scanlines
Depth
409
Graph Cuts
410
Graph Cuts
411
Multi-View Depth Reconstruction
work with Q. Yang, L. Wang, R. Yang

• Plane-sweep stereo on GPU

412
Middlebury Stereo Record
work with Q. Yang, L. Wang, R. Yang
Double-BP Highly computationally demanding even
for small images
Color-weighted correlation Real-time for small
images and few disparity levels
413
Depth Map Fusion

• Main lesson simple stereo with many correlations
  on many images fusion is the winning recipe

Depth map fusion
Highly optimized, computationally demanding
stereo
Simpler stereo on more data (higher number of
correlations)
414
GPU Stereo
GPU
CPU
415
GPU Stereo
GPU (NVIDIA 7800 GTX) 70ms
CPU (Xeon 3GHz) 3.2s
416
ICP
417
Alignment of Video onto 3D Point Clouds
work with Wen-Yi Zhao and Steve Hsu
Pose Estimation
Motion Stereo
ICP Alignment
418
Fusion

• Curless Levoy

419
Median Fusion
420
Stability Occlusion-Passing
Depth
421
(No Transcript)
422
Depth Map Fusion

• Resolves inconsistencies. Cleans up results very
  efficiently
• Suited for GPU implementation (essentially
  consists of rendering back and forth many times)

423
(No Transcript)
424
(No Transcript)
425
(No Transcript)
426
(No Transcript)
427
Depth Map Fusion
428
Sparse Mesh Generation
429
Computation times CPU
Single CPU processing times for single video
stream
Running the whole system with 1024x768
resolution for Radial, Tracker 2D, Tracker 3D,
Geo registration 512x384 resolution for Stereo,
Fusion, 3D model generation
seconds
430
Computation times CPUGPU
Single CPU GPU processing times for single
video stream
Running the whole system with 1024x768
resolution for Radial, Tracker 2D, Tracker 3D,
Geo registration 512x384 resolution for Stereo,
Fusion, 3D model generation
seconds
431
(No Transcript)
432
(No Transcript)
433
(No Transcript)
434
(No Transcript)
435
(No Transcript)
436
(No Transcript)
437
(No Transcript)
438
(No Transcript)
439
(No Transcript)
440
(No Transcript)
441
Camera Geometry

• Often leads to polynomial formulations,
• or can at least very often be formulated in
• terms of polynomial equations

442
Polynomial Formulation

• p1(x) , , pn(x) A set of input polynomials (n
  polynomials in m variables)
• xy1 ym

443
Algebraic Ideal

• I(p1 , , pn) The set of polynomials
• generated by the input polynomials
• (through additions and multiplications by a
  polynomial)
• p and q in I gt pq in I
• p in I gt pq in I

The ideal I consists of Almost all the
polynomials implied by the input
polynomials (More precisely, the radical
of the ideal consists of all)
444
Remember Row Operations

• Multiplying a row by a scalar
• Subtracting a row from another
• Swap rows

Add

• Multiplying a row by any polynomial

445
Multiplying by a Scalar
Transitions through zero remain
p(x)
3.8p(x)
446
Adding
Common transitions through zero remain
p1(x)
p2(x)
p1(x) p2(x)
447
Multiplying
Transitions through zero remain
p(x)f(x)
f(x)
p(x)
448
Basis (for Ideal)

• A basis for I is a set of polynomials
• (p1 , , pn) such that II(p1 , , pn)

449
Algebraic Variety

• The solution set
• (the vanishing set of the input polynomials)

V(I)xI(x)0
More precisely p(x)0 for all p in I
450
Quotient Ring J/I

• The set of equivalence classes of polynomials
  when only the values on V are considered (i.e.
  polynomials are equivalent iff p(x)q(x) for all
  x in V)

p in J/I
V(I)
451
Action Matrix

• For multiplication by polynomial on finite
  dimensional solution space

V(I)
452
Action Matrix
Action Matrix
Companion Matrix
453
An Equivalence
Compute Companion Matrix
Finding the Eigenvalues of a Matrix
Finding the Roots of a Polynomial
Compute Characteristic Polynomial
Requires Gröbner Basis for Input Equations
Compute Action Matrix in Quotient
Ring (Polynomials modulo Input Equations)
Finding the Roots of Multiple Polynomial
Equations
Finding the Eigenvalues of a Matrix
Compute Characteristic Polynomial
454
Companion Matrix
a7x7 a6x6 a5x5 a4x4 a3x3 a2x2 a1xa0
x2
1
x3
x4
x5
x6
455
Action Matrix
I
456
Action Matrix
I
457
Action Matrix
p in J
I
458
Action Matrix
p in J
I
p in J/I
459
Action Matrix
I
p in J/I
460
Action Matrix
I
q in J/I
p in J/I
461
Action Matrix
pq in J/I
I
q in J/I
p in J/I
462
Action Matrix
Multiplication by a polynomial q is a linear
operator Aq
(apßr)qa(pq)ß(rq)
The matrix Aq is called the action matrix for
multiplication by q
463
Action Matrix
I
464
Action Matrix
I
465
Action Matrix
I
466
Action Matrix
The values q(xi) of q at the solutions xi are the
eigenvalues of the action matrix
I
467
Action Matrix
The values q(xi) of q at the solutions xi are the
eigenvalues of the action matrix If we choose
qy1 , the eigenvalues are the solutions for y1
468
Action Matrix
br1 ro
b(x)Aq pq(x)b(x)p for all p in J/I and x in
V(I)
b(x)Aq b(x)q(x) b(x) is a left nullvector of
Aq corresponding to eigenvalue q(x)
469
Monomial Order

• Needed to define leading term of a polynomial
• Grevlex (Graded reverse lexicographical) order
  usually most efficient

y_2
y_1
470
Gröbner Basis

• A basis for ideal I that exposes the leading
  terms of I (hence unique well defined remainders)
• Easily gives the action matrix for multiplication
  with any polynomial in the quotient ring

y_2
y_1
471
A Reduced Gröbner Basis is a Basis in the normal
sense

• A polynomial in the ideal I can be written as a
  unique combination of the polynomials in a
  reduced Gröbner basis for I
• The monic Gröbner basis for I is unique

472
Buchbergers Algorithm
Buchbergers Algorithm
Euclids Algorithm for the GCD
Gaussian Elimination
473
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

474
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

475
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

476
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

477
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

478
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

479
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

480
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

481
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

482
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

483
Buchbergers Algorithm

• Compute remainders of S-polynomials until all
  remainders are zero

484
Prime Field Formulation

• Reals gt Cancellation unclear
• Rationals gt Grows unwieldy
• Prime Field gt Cancellation clear, size is
  limited, only small risk of incorrect
  cancellation if prime is large

485
Gaussian Elimination

• Expanding all polynomials up to a certain degree
  followed by Gaussian elimination allows pivoting

486
Unwanted Solutions
Can be removed by ideal quotients, or more
generally saturation
487
Elimination Example
488
Elimination Example
489
Elimination Example
490
Elimination Example
491
Elimination Example
492
Elimination Example
493
Elimination Example
494
Elimination Example
495
Elimination Example
496
Elimination Example
497
Action Matrix
498
(No Transcript)
499
Stratified Self-Calibration
Introduction
Camera calibration and the search for
infinity Hartley, Hayman, de Agapito, Reid
Calibration with robust use of cheirality by
quasi-affine reconstruction of the set of camera
projection centres Nister
500
Self-calibration
Pre-calibration
Less problems with critical surfaces (when
information used correctly)
Flexible
501
What is the cue in self-calibration?
502
(No Transcript)
503
(No Transcript)
504
(No Transcript)
505
Distortion of the cameras is the cue that drives
self- calibration
506
To move across the plane at infinity, a camera
has to go through a geometric wormhole
This makes the camera very angry and upset, in
fact it will refuse
507
Quasi-affine transformationsand cheirality
A projective transformation is quasi-affine with
respect to a set iff it preserves the convex hull
of the set
508
A projective transformation is affine iff it is
quasi-affine with respect to the set of all
finite points
509
(No Transcript)
510
(No Transcript)
511
(No Transcript)
512
(No Transcript)
513
Each camera pair poses a question regarding the
metric baseline
?
This or This
514
The question is easily answered by cheirality
since a point in front of or behind both cameras
supports the former case and a point on
different sides supports the latter.
A sequence of such binary decisions then deduces
the convex hull of the camera centres.
515
Using cheirality, the convex hull of the points
and the convex hull of the cameras can be
respected (But not necessarily the convex hull of
the union)
516
Metric configuration
517
(No Transcript)
518
(No Transcript)
519
Metric configuration
520
The points are not essential, convergence occurs
even from this projective equivalent