Photo Stitching Panoramas from Multiple Images - PowerPoint PPT Presentation

Loading...

PPT – Photo Stitching Panoramas from Multiple Images PowerPoint presentation | free to download - id: 5e1e02-YTQzZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Photo Stitching Panoramas from Multiple Images

Description:

04/06/10 Photo Stitching Panoramas from Multiple Images Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Bundle adjustment for stitching Non-linear ... – PowerPoint PPT presentation

Number of Views:220
Avg rating:3.0/5.0
Slides: 44
Provided by: derek73
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Photo Stitching Panoramas from Multiple Images


1
Photo Stitching Panoramas from Multiple Images
04/06/10
  • Computer Vision
  • CS 543 / ECE 549
  • University of Illinois
  • Derek Hoiem

2
So far, weve looked at what can be done with one
image
  • Recover basic geometry using vanishing points
  • Find image boundaries and segment objects
  • Categorize images
  • Find specific objects and detect objects that are
    part of some category

3
What can we get from multiple images?
4
What can we get from multiple images?
  • Bigger, Better, Brighter, Sharper images
  • Panoramas
  • Increased dynamic range
  • Super-resolution
  • Reduced noise/blur

Product example http//www.vreveal.com/
5
What can we get from multiple images?
  • Bigger, Better, Brighter, Sharper images
  • Panoramas
  • Increased dynamic range
  • Super-resolution
  • Reduced noise/blur

today
Product example http//www.vreveal.com/
6
What can we get from multiple images?
  • Depth and 3D structure
  • Two-view stereo
  • Multi-view stereo
  • Shape carving
  • Structure from motion

Thursday Next Tuesday
7
What can we get from multiple images?
  • Motion
  • Optical flow
  • Tracking
  • Action/activity recognition

Tracking (from Deva Ramanan)
Optical flow (source http//www.borisfx.com/avid/
bccavx/classic_features.php)
8
What can we get from multiple images?
  • Motion
  • Optical flow
  • Tracking
  • Action/activity recognition

April 15
Tracking (from Deva Ramanan)
Optical flow (source http//www.borisfx.com/avid/
bccavx/classic_features.php)
9
What can we get from multiple images?
  • Motion
  • Optical flow
  • Tracking
  • Action/activity recognition

April 20
Tracking (from Deva Ramanan)
Optical flow (source http//www.borisfx.com/avid/
bccavx/classic_features.php)
10
Today Image Stitching
  • Combine two or more overlapping images to make
    one larger image

Add example
Slide credit Vaibhav Vaish
11
Photo stitching
Camera Center
12
Example
Camera Center
13
Problem basics
  • Do on board

14
Basic problem
  • x K R t X
  • x K R t X
  • tt0
  • xHx where H K R R-1 K-1
  • Typically only R and f will change (4
    parameters), but, in general, H has 8 parameters

.
X
x
x'
f
f'
15
Image Stitching Algorithm Overview
  1. Detect keypoints
  2. Match keypoints
  3. Estimate homography with four matched keypoints
    (using RANSAC)
  4. Combine images

16
Computing homography
  • Assume we have four matched points How do we
    compute homography H?
  • Direct Linear Transformation (DLT)

Only these two provide unique constraints
17
Computing homography
  • Direct Linear Transform
  • Apply SVD UDVT A
  • h Vsmallest (column of V corr. to smallest
    singular value)

18
Computing homography
  • Assume we have four matched points How do we
    compute homography H?
  • Normalized DLT
  • Normalize coordinates for each image
  • Translate for zero mean
  • Scale so that average distance to origin is
    sqrt(2)
  • This makes problem better behaved numerically
    (see HZ p. 107-108)
  • Compute H using DLT in normalized coordinates
  • Unnormalize

19
Computing homography
  • Assume we have matched points with outliers How
    do we compute homography H?
  • Automatic Homography Estimation with RANSAC
  • Choose number of samples N

HZ Tutorial 99
20
Computing homography
  • Assume we have matched points with outliers How
    do we compute homography H?
  • Automatic Homography Estimation with RANSAC
  • Choose number of samples N
  • Choose 4 random potential matches
  • Compute H using normalized DLT
  • Project points from x to x for each potentially
    matching pair
  • Count points with projected distance lt t
  • t 6 s s is measurement error (1-3
    pixels)
  • Repeat steps 2-5 N times
  • Choose H with most inliers

HZ Tutorial 99
21
Automatic Image Stitching
  • Compute interest points on each image
  • Find candidate matches
  • Estimate homography H using matched points and
    RANSAC with normalized DLT
  • Transform second image and blend the two images
  • Matlab maketform, imtransform

Some details from a class project
22
Recognizing Panoramas
Brown and Lowe 2003, 2007
Some of following material from Brown and Lowe
2003 talk
23
Recognizing Panoramas
  • Input N images
  • Extract SIFT points, descriptors from all images
  • Find K-nearest neighbors for each point (K4)
  • For each image
  • Select M candidate matching images by counting
    matched keypoints (m6)
  • Solve homography Hij for each matched image

24
Recognizing Panoramas
  • Input N images
  • Extract SIFT points, descriptors from all images
  • Find K-nearest neighbors for each point (K4)
  • For each image
  • Select M candidate matching images by counting
    matched keypoints (m6)
  • Solve homography Hij for each matched image
  • Decide if match is valid (ni gt 8 0.3 nf )

features in overlapping area
inliers
25
RANSAC for Homography
Initial Matched Points
26
RANSAC for Homography
Final Matched Points
27
Verification
28
RANSAC for Homography
29
Recognizing Panoramas (cont.)
  • (now we have matched pairs of images)
  • Find connected components

30
Finding the panoramas
31
Finding the panoramas
32
Finding the panoramas
33
Recognizing Panoramas (cont.)
  • (now we have matched pairs of images)
  • Find connected components
  • For each connected component
  • Perform bundle adjustment to solve for rotation
    (?1, ?2, ?3) and focal length f of all cameras
  • Project to a surface (plane, cylinder, or sphere)
  • Render with multiband blending

34
Bundle adjustment for stitching
  • Non-linear minimization of re-projection error
  • where H K R R-1 K-1
  • Solve non-linear least squares (Levenberg-Marquard
    t algorithm)
  • See paper for details

35
Bundle Adjustment
  • New images initialised with rotation, focal
    length of best matching image

36
Bundle Adjustment
  • New images initialised with rotation, focal
    length of best matching image

37
Straightening
  • Rectify images so that up is vertical

38
Blending
  • Gain compensation minimize intensity difference
    of overlapping pixels
  • Blending
  • Pixels near center of image get more weight
  • Multiband blending to prevent blurring

39
Multi-band Blending
  • Burt Adelson 1983
  • Blend frequency bands over range ? l

40
Multiband blending
41
Blending comparison (IJCV 2007)
42
Blending Comparison
43
Further reading
  • DLT algorithm HZ p. 91 (alg 4.2), p. 585
  • Normalization HZ p. 107-109 (alg 4.2)
  • RANSAC HZ Sec 4.7, p. 123, alg 4.6
  • Recognising Panoramas Brown and Lowe, IJCV 2007
    (also bundle adjustment)

44
Things to remember
  • Homography relates rotating cameras
  • Recover homography using RANSAC and normalized
    DLT
  • Bundle adjustment minimizes reprojection error
    for set of related images
  • Details to make it look nice (straightening,
    blending)

45
Next class
  • How to relate images taken from cameras with
    different centers
About PowerShow.com