Real-time Acquisition and Rendering of Large 3D Models - PowerPoint PPT Presentation

About This Presentation
Title:

Real-time Acquisition and Rendering of Large 3D Models

Description:

... pieces into single coordinate frame: initial manual alignment, refined with ... Table lookup based on illumination history and position in four-frame sequence ... – PowerPoint PPT presentation

Number of Views:113
Avg rating:3.0/5.0
Slides: 64
Provided by: szymonrus
Category:

less

Transcript and Presenter's Notes

Title: Real-time Acquisition and Rendering of Large 3D Models


1
Real-time Acquisition and Rendering of Large 3D
Models
  • Szymon Rusinkiewicz

2
Computer Graphics Pipeline
Shape
Motion
Lighting and Reflectance
  • Human time expensive
  • Sensors cheap
  • Computer graphics increasingly relies
    onmeasurements of the real world

3
3D Scanning Applications
  • Computer graphics
  • Product inspection
  • Robot navigation
  • As-built floorplans
  • Product design
  • Archaeology
  • Clothes fitting
  • Art history

4
The Digital Michelangelo Project
  • Push state of the art in range scanning and
    demonstrate applications in art and art history

Working in the museum
Scanning geometry
Scanning color
5
Traditional Range Scanning Pipeline
  • High-quality, robust pipeline for producing 3D
    models
  • Scan object with laser triangulation scanner
    many views from different angles
  • Align pieces into single coordinate
    frameinitial manual alignment, refined with ICP
  • Merge overlapping regions compute average
    surface using VRIP Curless Levoy 96
  • Display resulting model

6
3D Scan of David Statistics
  • Over 5 meters tall
  • 1/4 mm resolution
  • 22 people
  • 30 nights of scanning
  • Efficiency max min 8 1
  • Needed view planning
  • Weight of gantry 800 kg
  • Putting model together1000 man-hours and
    counting

7
New 3D Scanning Pipeline
  • Need for a fast, inexpensive,easy-to-use 3D
    scanning system
  • Wave a (small, rigid) object by hand in front of
    the scanner
  • Automatically align data asit is acquired
  • Let user see partial model as itis being built
    fill holes

8
Real-Time 3D Model Acquisition
  • Prototype real-time model acquisition system
  • 3D scanning of moving objects
  • Fast alignment
  • Real-time merging and display

9
Applications of Easy-to-Use3D Model Acquisition
  • Advertising
  • More capabilities in Photoshop
  • Movie sets
  • Augmented reality
  • User interfaces

10
3D Scanning Technologies
  • Contact-based touch probes
  • Passive shape from stereo, motion, shading
  • Active time-of-flight, defocus, photometric
    stereo, triangulation
  • Triangulation systems are inexpensive, robust,
    and flexible
  • Take advantage of trends in DLP projectors

11
Laser Triangulation
Object
  • Project laser stripe onto object

12
Laser Triangulation
Object
(x,y)
  • Depth from ray-plane triangulation

13
Triangulation
  • Faster acquisition project multiple stripes
  • Correspondence problem which stripeis which?

14
Triangulation
Slow, robust
Fast, fragile
15
Time-Coded Light Patterns
  • Assign each stripe a unique illumination
    codeover time Posdamer 82

Time
Space
16
Gray-Code Patterns
  • To minimize effects of quantization erroreach
    point may be a boundary only once

Time
Space
17
Structured-Light Assumptions
  • Structured-light systems make certain assumptions
    about the scene
  • Spatial continuity assumption
  • Assume scene is one object
  • Project a grid, pattern of dots, etc.
  • Temporal continuity assumption
  • Assume scene is static
  • Assign stripes a code over time

18
Codes for Moving Scenes
  • We make a different assumption
  • Object may move
  • Velocity low enough to permit tracking
  • Spatio-temporal continuity

19
Codes for Moving Scenes
  • Code stripe boundariesinstead of stripes
  • Perform frame-to-frametracking of
    correspondingboundaries
  • Propagate illumination history
  • Hall-Holt Rusinkiewicz, ICCV 2001

20
New Scanning Pipeline
Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
21
Designing a Code
Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
  • Biggest problem is ghosts WW or BB boundaries
    that cant be seen directly

22
Designing a Code
  • Design a code to make tracking possible
  • Do not allow two spatially adjacent ghosts
  • Do not allow two temporally adjacent ghosts

t
23
Designing a Code
  • Graph (for 4 frames)

24
Designing a Code
  • Graph (for 4 frames)
  • Edges boundaries (over time)
  • Path with alternating colors55 edges in graph
    ?maximal-length traversal has 110 boundaries
    (111 stripes)

25
Image Capture
Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
  • Standard video camera fields at 60 Hz
  • Genlock camera to projector

26
Finding Boundaries
Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
  • Standard edge detection problem
  • Current solution find minima and maxima of
    intensity, boundary is between them

27
Matching Stripe Boundaries
Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
  • Even if number of ghosts isminimized, matching
    is not easy

?
28
Matching Stripe Boundaries
  • Resolve ambiguity by constraining maximum stripe
    velocity
  • Could accommodate higher speeds by estimating
    velocities
  • Could take advantage of methods intracking
    literature (e.g., Kalman filters)

29
Decoding Boundaries
Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
  • Propagate illumination history
  • Table lookup based on illumination history and
    position in four-frame sequence
  • Once a stripe has been tracked for at least four
    frames,it contributes useful data on every
    subsequent frame

30
Computing 3D Position
Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
  • Ray-plane intersection
  • Requires calibration of
  • Camera, projector intrinsics
  • Relative position and orientation

31
Results
Video frames
Stripe boundaries
unknown known ghosts
32
Results
  • Single range image of moving object

Top View
Top View
Front View
Front View
Boundary codes and tracking
Gray codes, no tracking
33
Aligning 3D Data
  • This range scanner can be used for any moving
    objects
  • For rigid objects, range images can be aligned to
    each other as object moves

34
Aligning 3D Data
  • If correct correspondences are known,it is
    possible to find correct relative
    rotation/translation

35
Aligning 3D Data
  • How to find corresponding points?
  • Previous systems based on user input,feature
    matching, surface signatures, etc.

36
Aligning 3D Data
  • Alternative assume closest points correspond to
    each other, compute the best transform

37
Aligning 3D Data
  • and iterate to find alignment
  • Iterated Closest Points (ICP) Besl McKay 92
  • Converges if starting position close enough

38
ICP Variants
  • Classic ICP algorithm not real-time
  • To improve speed examine stages of ICP and
    evaluate proposed variants
  • Rusinkiewicz Levoy, 3DIM 2001
  1. Selecting source points (from one or both meshes)
  2. Matching to points in the other mesh
  3. Weighting the correspondences
  4. Rejecting certain (outlier) point pairs
  5. Assigning an error metric to the current
    transform
  6. Minimizing the error metric

39
ICP Variant Point-to-Plane Error Metric
  • Using point-to-plane distance instead of
    point-to-point lets flat regions slide along each
    other more easily Chen Medioni 91

40
Finding Corresponding Points
  • Finding closest point is most expensive stage of
    ICP
  • Brute force search O(n)
  • Spatial data structure (e.g., k-d tree) O(log
    n)
  • Voxel grid O(1), but large constant, slow
    preprocessing

41
Finding Corresponding Points
  • For range images, simply project point Blais 95
  • Constant-time, fast
  • Does not require precomputing a spatial data
    structure

42
High-Speed ICP Algorithm
  • ICP algorithm with projection-based
    correspondences, point-to-plane matchingcan
    align meshes in a few tens of ms.(cf. over 1
    sec. with closest-point)

43
Anchor Scans
  • Alignment of consecutive scans leads to
    accumulation of ICP errors
  • Alternative align all scans to an anchor scan,
    only switch anchor when overlap low
  • Given anchor scans, restart after failed ICP
    becomes easier

44
Merging and Rendering
  • Goal visualize the model well enoughto be able
    to see holes
  • Cannot display all the scanned data accumulates
    linearly with time
  • Standard high-quality merging methodsprocessing
    time 1 minute per scan

45
Merging and Rendering
  • Real-time incremental merging and rendering
  • Quantize samples to a 3D grid
  • Maintain average normal of all pointsat a grid
    cell
  • Point (splat) rendering
  • Can be made hierarchical to conserve memory

46
Photograph
47
Real-time Scanning Demo

48
Postprocessing
  • Goal of real-time display is to let user evaluate
    coverage, fill holes
  • Quality/speed tradeoff
  • Offline postprocessing for high-quality models

49
Merged Result
Photograph
Aligned scans
Merged
50
Future Work
  • Technological improvements
  • Use full resolution of projector
  • Higher-resolution cameras
  • Ideas from design of single-stripe 3D scanners
  • Pipeline improvements
  • Better detection of failed alignment
  • Better handling of object texture combine with
    stereo?
  • Global registration to eliminate drift
  • More sophisticated merging
  • Improve user interaction during scanning

51
Future Work
  • Faster scanning
  • Better stripe boundary matching
  • Multiple cameras, projectors
  • High-speed cameras
  • Application in different contexts
  • Small, hand-held
  • Cart- or shoulder-mounted for digitizing rooms
  • Infrared for imperceptibility

52
Rendering of Large Models
  • Range scanners increasingly capable of producing
    very large models
  • DMich models are 100 million to 1 billion samples
  • Challenge how to allow viewing in real time
  • Fast startup, progressive loading
  • Traditional answer triangle meshes,
    simplification, hardware-accelerated rendering
  • Impractical for such large models
  • Alternative revisit basic data structure

53
QSplat
  • Key observation a single bounding sphere
    hierarchy can be used for
  • Hierarchical frustum and backface culling
  • Level of detail control
  • Splat rendering Westover 89

54
QSplat Node Structure
Position and Radius
Width of Cone of Normals
Tree Structure
Color (Optional)
Normal
13 bits
3 bits
14 bits
2 bits
16 bits
6 bytes
55
QSplat Node Structure
Position and Radius
Width of Cone of Normals
Tree Structure
Color (Optional)
Normal
13 bits
3 bits
14 bits
16 bits
2 bits
  • Position and radius encoded relative to parent
    node
  • Hierarchical coding vs. delta coding along a path
    for vertex positions

56
QSplat Node Structure
Position and Radius
Width of Cone of Normals
Tree Structure
Color (Optional)
Normal
13 bits
3 bits
14 bits
16 bits
2 bits
57
QSplat Node Structure
Position and Radius
Width of Cone of Normals
Tree Structure
Color (Optional)
Normal
13 bits
3 bits
14 bits
16 bits
2 bits
58
QSplat Node Structure
Position and Radius
Width of Cone of Normals
Tree Structure
Color (Optional)
Normal
13 bits
3 bits
14 bits
16 bits
2 bits
59
QSplat Rendering Algorithm
  • Traverse hierarchy recursively

if (node not visible) Skip this branch else if
(leaf node) Draw a splat else if (size on screen
lt threshold) Draw a splat else Traverse children
60
Demo St. Matthew
  • 3D scan of 2.7 meter statue at 0.25 mm
  • 102,868,637 points
  • File size 644 MB
  • Preprocessing time 1 hour

61
Future Work
  • Splats as primitive
  • Unify rendering of meshes, volumes, point clouds
  • Compatible with shading after rasterization
  • Hybrid point/polygon systems
  • High-level visibility / LOD frameworks
  • Store different kinds of data at each node
    alpha, BRDF, scattering function, etc.
  • Potentially could be used to unify
    image-based-rendering (IBR) techniques

62
Contributions
  • Real-time 3D model acquisition system
  • Video-rate 3D scanner for moving objects
  • Analysis of ICP variants real-time algorithm
  • Real-time merging and rendering
  • Allows user to see model and fill holes
  • QSplat interactive rendering of large 3D meshes
  • Single data structure used for visibility
    culling,level-of-detail control, point
    rendering, compression
  • Extension to network streaming I3D 2001

63
Acknowledgments
  • Olaf Hall-Holt
  • Lucas Pereira
  • The Original DMich Gang Dave Koller, Sean
    Anderson, James Davis, Kari Pulli, Matt Ginzton,
    Jon Shade
  • DMich, the next generation Gary King, Steve
    Marschner
  • Graphics lab
  • Advisor Marc Levoy
  • Committee Pat Hanrahan, Leo Guibas, Mark
    Horowitz, Bernd Girod
  • Family, friends
  • Sponsors NSF, Interval, Honda, Sony, Intel
Write a Comment
User Comments (0)
About PowerShow.com