Real-Time 3D Model Acquisition

1
Real-Time 3D Model Acquisition
Princeton University Stanford University

• Szymon Rusinkiewicz
• Olaf Hall-Holt
• Marc Levoy

2
3D Scanning
3
Possible Research Goals

• Low noise
• Guaranteed high accuracy
• High speed
• Low cost
• Automatic operation
• No holes

4
3D Model Acquisition Pipeline
3D Scanner
5
3D Model Acquisition Pipeline
3D Scanner
6
3D Model Acquisition Pipeline
3D Scanner
7
3D Model Acquisition Pipeline
3D Scanner
8
3D Model Acquisition Pipeline
3D Scanner
9
3D Model Acquisition Pipeline
3D Scanner
10
3D Model Acquisition Difficulties

• Much (often most) time spent on last 20
• Pipeline not optimized for hole-filling
• Not sufficient just to speed up scanner must
  design pipeline for fast feedback

11
Real-Time 3D Model Acquisition
12
Real-Time 3D Model Acquisition Pipeline
3D Scanner
Alignment
View Planning
Human
Merging
Done?
Display
13
Real-Time 3D Model Acquisition Pipeline
3D Scanner
View Planning
Challenge Real Time
Done?
Display
14
Real-Time 3D Model Acquisition Pipeline
3D Scanner
Alignment
View Planning
Part I Structured-LightTriangulation
Merging
Done?
Display
15
Real-Time 3D Model Acquisition Pipeline
3D Scanner
Alignment
View Planning
Part II Fast ICP
Merging
Done?
Display
16
Real-Time 3D Model Acquisition Pipeline
3D Scanner
Alignment
View Planning
Part III Voxel Grid
Merging
Done?
Display
17
Triangulation
Object

• Project laser stripe onto object

18
Triangulation
Object
Laser
(x,y)

• Depth from ray-plane triangulation

19
Triangulation

• Faster acquisition project multiple stripes
• Correspondence problem which stripeis which?

20
Continuum of Triangulation Methods
Slow, robust
Fast, fragile
21
Time-Coded Light Patterns

• Assign each stripe a unique illumination
  codeover time Posdamer 82

Time
Space
22
Codes for Moving Scenes

• Assign time codesto stripe boundaries
• Perform frame-to-frametracking of
  correspondingboundaries
• Propagate illumination history
• Hall-Holt Rusinkiewicz, ICCV 2001

23
Designing a Code

• Want many features to tracklots of
  black/white edges at each frame
• Try to minimize ghosts WW or BB boundaries
  that cant be seen directly

24
Designing a Code
0000
1101
1010
0111
1111
0010
1000
0101
1011
0110
0001
1100
0100
1001
0011
1110
Hall-Holt Rusinkiewicz, ICCV 2001
25
Implementation

• Pipeline
• DLP projector illuminates scene _at_ 60 Hz.
• Synchronized NTSC camera captures video
• Pipeline returns range images _at_ 60 Hz.

Project Code
Capture Images
Find Boundaries
Match Boundaries
Decode
Compute Range
26
Real-Time 3D Model Acquisition Pipeline
3D Scanner
Alignment
View Planning
Part II Fast ICP
Merging
Done?
Display
27
Aligning 3D Data

• This range scanner can be used for anymoving
  objects
• For rigid objects, range images can be aligned to
  each other as object moves

28
Aligning 3D Data

• ICP (Iterative Closest Points) for each point on
  one scan, minimize distance to closest point on
  other scan

29
Aligning 3D Data

• and iterate to find alignment
• Iterated Closest Points (ICP) Besl McKay 92

30
ICP in the Real-Time Pipeline

• Potential problem with ICP local minima
• In this pipeline, scans close together
• Very likely to converge to correct (global)
  minimum
• Basic ICP algorithm too slow ( seconds)
• Point-to-plane minimization
• Projection-based matching
• With these tweaks, running time
  millisecondsRusinkiewicz Levoy, 3DIM 2001

31
Real-Time 3D Model Acquisition Pipeline
3D Scanner
Alignment
View Planning
Part III Voxel Grid
Merging
Done?
Display
32
Merging and Rendering

• Goal visualize the model well enoughto be able
  to see holes
• Cannot display all the scanned data accumulates
  linearly with time
• Standard high-quality merging methodsprocessing
  time 1 minute per scan

33
Merging and Rendering
34
Merging and Rendering
35
Merging and Rendering
36
Merging and Rendering
37
Merging and Rendering

• Point rendering, using accumulated normals for
  lighting

38
Example Photograph
18 cm.
39
Result
40
Postprocessing

• Real-time display
• Quality/speed tradeoff
• Goal let user evaluate coverage, fill holes
• Offline postprocessing for high-quality models
• Global registration
• High-quality merging (e.g., using VRIP Curless
  96)

41
Postprocessed Model
42
Recapturing Alignment
43
Summary

• 3D model acquisition pipeline optimized for
  obtaining complete, hole-free models
• Use humans time most efficiently
• Pieces of pipeline selected for real-time use
• Structured-light scanner for moving objects
• Fast ICP variant
• Simple grid-based merging, point rendering

44
Limitations

• Prototype noisier than commercial systems
• Could be made equivalent with careful engineering
• Ultimate limitations on quality focus, texture
• Scan-to-scan ICP not perfect ? alignment drift
• Due to noise, miscalibration, degenerate geometry
• Reduced, but not eliminated, by anchor scans
• Possibly combine ICP with separate trackers

45
Future Work

• Faster scanning
• Better stripe boundary tracking
• Multiple cameras, projectors
• High-speed cameras, projectors
• Application in different contexts
• Cart- or shoulder-mounted for digitizing rooms
• Infrared for imperceptibility

46
Acknowledgments

• Collaborators
• Li-Wei He
• James Davis
• Lucas Pereira
• Sean Anderson
• Sponsors
• Sony
• Intel
• Interval