Computational Photography: Image-based Modeling - PowerPoint PPT Presentation

Loading...

PPT – Computational Photography: Image-based Modeling PowerPoint presentation | free to download - id: 6ee87a-OTZkN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Computational Photography: Image-based Modeling

Description:

Title: Computer Vision: Multiview Stereo Author: Steve Seitz Last modified by: jchai Created Date: 4/21/1999 5:01:05 AM Document presentation format – PowerPoint PPT presentation

Number of Views:70
Avg rating:3.0/5.0
Slides: 39
Provided by: SteveS231
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Computational Photography: Image-based Modeling


1
Computational PhotographyImage-based Modeling
  • Jinxiang Chai

2
Image-based modeling
  • Estimating 3D structure
  • Estimating motion, e.g., camera motion
  • Estimating lighting
  • Estimating surface model

3
Traditional modeling and rendering
Geometry Reflectance Light source Camera model
rendering
User input Texture map survey data
modeling
Images
For photorealism - Modeling is hard -
Rendering is slow
4
Can we model and render this? What do we want to
do for this model?
5
Image based modeling and rendering
Image-based modeling
Image-based rendering
Images user input range scans
Model
Images
6
Spectrum of IBMR
Model
Panoroma
Image-based rendering
Image based modeling
Images Depth
Geometry Images
Camera geometry
Images user input range scans
Images
Geometry Materials
Light field
Kinematics
Dynamics
Etc.
7
Spectrum of IBMR
Model
Panoroma
Image-based rendering
Image based modeling
Images Depth
Geometry Images
Camera geometry
Images user input range scans
Images
Geometry Materials
Light field
Kinematics
Dynamics
Etc.
8
Spectrum of IBMR
Model
Panoroma
Image-based rendering
Image based modeling
Images Depth
Geometry Images
Camera geometry
Images user input range scans
Images
Geometry Materials
Light field
Kinematics
Dynamics
Etc.
9
Stereo Reconstruction
  • Given two or more images of the same scene or
    object, compute a representation of its shape
  • What are some possible applications?

known camera viewpoints
10
3D Modeling
  • From one stereo pair to a 3D head model
  • Frederic Deverney, INRIA

11
3D Modeling
The Digital Michelangelo Project, Levoy et al.
12
Optical mocap
Vicon mocap system
13
Z-keying mix live and synthetic
  • Takeo Kanade, CMU (Stereo Machine)

14
Virtualized RealityTM
  • Takeo Kanade et al., CMU
  • collect video from 50 stream
  • reconstruct 3D model sequences
  • steerable version used forSuperBowl XXV eye
    vision
  • http//www.cs.cmu.edu/afs/cs/project/VirtualizedR/
    www/VirtualizedR.html

15
View interpolation
  • input depth image novel view
  • Szeliski Kang 95

16
View morphing
  • Morph between pair of images using epipolar
    geometry Seitz Dyer, SIGGRAPH96

17
Image warping
18
Video view interpolation
19
Performance Interface
  • Microsoft Natal project

20
Additional applications?
  • Real-time people tracking (systems from Pt. Gray
    Research and SRI)
  • Gaze correction for video conferencing
    Ott,Lewis,Cox InterChi93
  • Other ideas?

21
Stereo matching
  • Given two or more images of the same scene or
    object, compute a representation of its shape
  • What are some possible representations for
    shapes?
  • depth maps
  • volumetric models
  • 3D surface models
  • planar (or offset) layers

22
Outline
  • Stereo matching
  • - Traditional stereo
  • Volumetric stereo
  • - Visual hull
  • - Voxel coloring

23
Papers
  • Stereo matching
  • Masatoshi Okutomi and Takeo Kanade. A
    multiple-baseline stereo. IEEE Trans. on Pattern
    Analysis and Machine Intelligence (PAMI), 15(4),
    1993, pp. 353--363.
  • D. Scharstein and R. Szeliski. A taxonomy and
    evaluation of dense two-frame stereo
    correspondence algorithms.International Journal
    of Computer Vision, 47(1/2/3)7-42, April-June
    2002.
  • Visual-hull reconstruction
  • Szeliski, Rapid Octree Construction from Image
    Sequences, Computer Vision, Graphics, and Image
    Processing Image Understanding, 58(1), 1993, pp.
    23-32.
  • Matusik, Buehler, Raskar, McMillan, and Gortler ,
    Image-Based Visual Hulls, Proc. SIGGRAPH 2000,
    pp. 369-374.

24
Stereo
scene point
image plane
optical center
25
Stereo
  • Basic Principle Triangulation
  • Gives reconstruction as intersection of two rays
  • Requires
  • calibration
  • point correspondence

26
Camera calibration
  • From world coordinate to image coordinate

Perspective projection
View transformation
Viewport projection
u0
sx
a
u
v0
-sy
0
v
1
0
0
1
Camera parameters
3D points
2D projections
27
Stereo correspondence
  • Determine Pixel Correspondence
  • Pairs of points that correspond to same scene
    point

epipolar line
  • Epipolar Constraint
  • Reduces correspondence problem to 1D search along
    conjugate epipolar lines
  • Java demo http//www.ai.sri.com/luong/research/
    Meta3DViewer/EpipolarGeo.html

28
Stereo image rectification
29
Stereo image rectification
  • reproject image planes onto a common
  • plane parallel to the line between optical
    centers
  • pixel motion is horizontal after this
    transformation
  • two homographies (3x3 transform), one for each
    input image reprojection
  • C. Loop and Z. Zhang. Computing Rectifying
    Homographies for Stereo Vision. IEEE Conf.
    Computer Vision and Pattern Recognition, 1999.

30
Rectification
Original image pairs
Rectified image pairs
31
Stereo matching algorithms
  • Match Pixels in Conjugate Epipolar Lines
  • Assume brightness constancy
  • This is a tough problem
  • Numerous approaches
  • A good survey and evaluation http//www.middlebu
    ry.edu/stereo/

32
Your basic stereo algorithm
  • compare with every pixel on same epipolar line in
    right image
  • pick pixel with minimum matching cost

33
Window size
Effect of window size
  • Smaller window
  • -
  • Larger window
  • -

34
Stereo results
  • Data from University of Tsukuba
  • Similar results on other images without ground
    truth

Ground truth
Scene
35
Results with window search
Window-based matching (best window size)
Ground truth
36
Better methods exist...
State of the art method Boykov et al., Fast
Approximate Energy Minimization via Graph Cuts,
International Conference on Computer Vision,
September 1999.
Ground truth
37
Stereo reconstruction pipeline
  • Steps
  • Calibrate cameras
  • Rectify images
  • Compute disparity
  • Estimate depth

38
Stereo reconstruction pipeline
  • Steps
  • Calibrate cameras
  • Rectify images
  • Compute disparity
  • Estimate depth
  • Camera calibration errors
  • Poor image resolution
  • Occlusions
  • Violations of brightness constancy (specular
    reflections)
  • Large motions
  • Low-contrast image regions

What will cause errors?
About PowerShow.com