CSCE 641 Computer Graphics: Image-based Modeling - PowerPoint PPT Presentation

Loading...

PPT – CSCE 641 Computer Graphics: Image-based Modeling PowerPoint presentation | free to download - id: 6ee877-YTZkN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

CSCE 641 Computer Graphics: Image-based Modeling

Description:

Title: Computer Vision: Multiview Stereo Author: Steve Seitz Last modified by: jchai Created Date: 4/21/1999 5:01:05 AM Document presentation format – PowerPoint PPT presentation

Number of Views:178
Avg rating:3.0/5.0
Slides: 59
Provided by: SteveS232
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: CSCE 641 Computer Graphics: Image-based Modeling


1
CSCE 641 Computer Graphics Image-based Modeling
  • Jinxiang Chai

2
Image-based modeling
  • Estimating 3D structure
  • Estimating motion, e.g., camera motion
  • Estimating lighting
  • Estimating surface model

3
Traditional modeling and rendering
Geometry Reflectance Light source Camera model
rendering
User input Texture map survey data
modeling
Images
For photorealism - Modeling is hard -
Rendering is slow
4
Can we model and render this? What do we want to
do for this model?
5
Image based modeling and rendering
Image-based modeling
Image-based rendering
Images user input range scans
Model
Images
6
Spectrum of IBMR
Model
Panoroma
Image-based rendering
Image based modeling
Images Depth
Geometry Images
Camera geometry
Images user input range scans
Images
Geometry Materials
Light field
Kinematics
Dynamics
Etc.
7
Spectrum of IBMR
Model
Panoroma
Image-based rendering
Image based modeling
Images Depth
Geometry Images
Camera geometry
Images user input range scans
Images
Geometry Materials
Light field
Kinematics
Dynamics
Etc.
8
Spectrum of IBMR
Model
Panoroma
Image-based rendering
Image based modeling
Images Depth
Geometry Images
Camera geometry
Images user input range scans
Images
Geometry Materials
Light field
Kinematics
Dynamics
Etc.
9
Stereo reconstruction
  • Given two or more images of the same scene or
    object, compute a representation of its shape
  • How can we estimate camera parameters?

known camera viewpoints
10
How can we estimate the camera parameters?
11
Camera calibration
  • Augmented pin-hole camera
  • - focal point, orientation
  • - focal length, aspect ratio, center, lens
    distortion

Known 3D
Classical calibration - 3D 2D -
correspondence
Camera calibration online resources
12
Camera and calibration target
13
Classical camera calibration
  • Known 3D coordinates and 2D coordinates
  • - known 3D points on calibration targets
  • - find corresponding 2D points in image
    using feature detection
  • algorithm

14
Camera parameters
Known 3D coords and 2D coords
u0
sx
?
u
v0
-sy
0
v
1
0
0
1
Perspective proj.
Viewport proj.
View trans.
15
Camera parameters
Known 3D coords and 2D coords
u0
sx
?
u
v0
-sy
0
v
1
0
0
1
Perspective proj.
Viewport proj.
View trans.
Intrinsic camera parameters (5 parameters)
extrinsic camera parameters (6 parameters)
16
Camera matrix
  • Fold intrinsic calibration matrix K and extrinsic
    pose parameters (R,t) together into acamera
    matrix
  • M K R t
  • (put 1 in lower r.h. corner for 11 d.o.f.)

17
Camera matrix calibration
  • Directly estimate 11 unknowns in the M matrix
    using known 3D points (Xi,Yi,Zi) and measured
    feature positions (ui,vi)

18
Camera matrix calibration
  • Linear regression
  • Bring denominator over, solve set of
    (over-determined) linear equations. How?

19
Camera matrix calibration
  • Linear regression
  • Bring denominator over, solve set of
    (over-determined) linear equations. How?
  • Least squares (pseudo-inverse)
  • - 11 unknowns (up to scale)
  • - 2 equations per point (homogeneous
    coordinates)
  • - 6 points are sufficient

20
Nonlinear camera calibration
  • Perspective projection

21
Nonlinear camera calibration
  • Perspective projection

K
R
T
P
22
Nonlinear camera calibration
  • Perspective projection
  • 2D coordinates are just a nonlinear function of
    its 3D coordinates and camera parameters

K
R
T
P
23
Nonlinear camera calibration
  • Perspective projection
  • 2D coordinates are just a nonlinear function of
    its 3D coordinates and camera parameters

K
R
T
P
24
Multiple calibration images
  • Find camera parameters which satisfy the
    constraints from M images, N points
  • for j1,,M
  • for i1,,N
  • This can be formulated as a nonlinear
    optimization problem

25
Multiple calibration images
  • Find camera parameters which satisfy the
    constraints from M images, N points
  • for j1,,M
  • for i1,,N
  • This can be formulated as a nonlinear
    optimization problem

Solve the optimization using nonlinear
optimization techniques - Gauss-newton -
Levenberg-Marquardt
26
Nonlinear approach
  • Advantages
  • can solve for more than one camera pose at a time
  • fewer degrees of freedom than linear approach
  • Standard technique in photogrammetry, computer
    vision, computer graphics
  • - Tsai 87 also estimates lens distortions
    (freeware _at_ CMU)http//www.cs.cmu.edu/afs/cs/proj
    ect/cil/ftp/html/v-source.html
  • Disadvantages
  • more complex update rules
  • need a good initialization (recover K R t
    from M)

27
Camera Calibration
  • Public calibration toolbox
  • - http//research.microsoft.com/en-us/um/people/zh
    ang/Calib/
  • - http//www.vision.caltech.edu/bouguetj/calib_doc
    /

28
How can we estimate the camera parameters?
29
Application camera calibration for sports video
images
Court model
Farin et. Al
30
Stereo matching
  • Given two or more images of the same scene or
    object as well as their camera parameters, how to
    compute a representation of its shape?
  • What are some possible representations for
    shapes?
  • depth maps
  • volumetric models
  • 3D surface models
  • planar (or offset) layers

31
Outline
  • Stereo matching
  • - Traditional stereo
  • - Active stereo
  • Volumetric stereo
  • - Visual hull
  • - Voxel coloring
  • - Space carving

32
Readings
  • Stereo matching
  • 11.1, 11.2,.11.3,11.5 in Sezliski book
  • D. Scharstein and R. Szeliski. A taxonomy and
    evaluation of dense two-frame stereo
    correspondence algorithms.International Journal
    of Computer Vision, 47(1/2/3)7-42, April-June
    2002.

33
Stereo
scene point
image plane
optical center
34
Stereo
  • Basic Principle Triangulation
  • Gives reconstruction as intersection of two rays
  • Requires
  • calibration
  • point correspondence

35
Stereo correspondence
  • Determine Pixel Correspondence
  • Pairs of points that correspond to same scene
    point

epipolar line
  • Epipolar Constraint
  • Reduces correspondence problem to 1D search along
    conjugate epipolar lines
  • Java demo http//www.ai.sri.com/luong/research/
    Meta3DViewer/EpipolarGeo.html

36
Stereo image rectification
37
Stereo image rectification
  • reproject image planes onto a common
  • plane parallel to the line between optical
    centers
  • pixel motion is horizontal after this
    transformation
  • two homographies (3x3 transform), one for each
    input image reprojection
  • C. Loop and Z. Zhang. Computing Rectifying
    Homographies for Stereo Vision. IEEE Conf.
    Computer Vision and Pattern Recognition, 1999.

38
Rectification
Original image pairs
Rectified image pairs
39
Stereo matching algorithms
  • Match Pixels in Conjugate Epipolar Lines
  • Assume brightness constancy
  • This is a tough problem
  • Numerous approaches
  • A good survey and evaluation http//www.middlebu
    ry.edu/stereo/

40
Your basic stereo algorithm
  • compare with every pixel on same epipolar line in
    right image
  • pick pixel with minimum matching cost

41
Window size
Effect of window size
  • Smaller window
  • -
  • Larger window
  • -

42
More constraints?
  • We can enforce more constraints to reduce
    matching ambiguity
  • - smoothness constraints computed disparity
    at a pixel
  • should be consistent with neighbors in a
    surrounding window.
  • - uniqueness constraints the matching needs
    to be bijective
  • - ordering constraints e.g., computed
    disparity at a pixel
  • should not be larger than the disparity of
    its right neighbor pixel by
  • more than one pixel.

43
Stereo results
  • Data from University of Tsukuba
  • Similar results on other images without ground
    truth

Ground truth
Scene
44
Results with window search
Window-based matching (best window size)
Ground truth
45
Better methods exist...
A better method Boykov et al., Fast Approximate
Energy Minimization via Graph Cuts,
International Conference on Computer Vision,
September 1999.
Ground truth
46
More recent development
  • High-Quality Single-Shot Capture of Facial
    Geometry siggraph 2010, project website
  • - capture high-fidelity facial geometry from
    multiple cameras
  • - pairwise stereo reconstruction between
    neighboring cameras
  • - hallucinate facial details

47
More recent development
  • High Resolution Passive Facial Performance
    Capture siggraph 2010, project website
  • - capture dynamic facial geometry from
    multiple video cameras
  • - spatial stereo reconstruction for every
    frame
  • - building temporal correspondences across
    the entire sequence

48
Stereo reconstruction pipeline
  • Steps
  • Calibrate cameras
  • Rectify images
  • Compute disparity
  • Estimate depth

49
Stereo reconstruction pipeline
  • Steps
  • Calibrate cameras
  • Rectify images
  • Compute disparity
  • Estimate depth
  • Camera calibration errors
  • Poor image resolution
  • Occlusions
  • Violations of brightness constancy (specular
    reflections)
  • Large motions
  • Low-contrast image regions

What will cause errors?
50
Outline
  • Stereo matching
  • - Traditional stereo
  • - Active stereo
  • Volumetric stereo
  • - Visual hull
  • - Voxel coloring
  • - Space carving

51
Active stereo with structured light
Li Zhangs one-shot stereo
  • Project structured light patterns onto the
    object
  • simplifies the correspondence problem

52
Active stereo with structured light
53
Laser scanning
Digital Michelangelo Project http//graphics.stanf
ord.edu/projects/mich/
  • Optical triangulation
  • Project a single stripe of laser light
  • Scan it across the surface of the object
  • This is a very precise version of structured
    light scanning

54
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
55
Laser scanned models
The Digital Michelangelo Project, Levoy et al.
56
RGBD Sensors
2010
2008
2013
Lower size/cost with better accuracy
57
Kinect Sensor
58
Kinect Sensor
About PowerShow.com