Efficiently Combining Positions and Normals for Precise 3D Geometry

1
Efficiently Combining Positions and Normals for
Precise 3D Geometry
2
Objects of interest
"Greek" panel
"American" vase
digital photographs
3
Measuring positions(stereo triangulation)
4
Measuring normals(photometric stereo)
5
Scanning results panel
triangulation normals
photometric normals
6
Scanning results vase
triangulation normals
photometric normals
7
Normal mapping is not enough
accessibility shading
rendering at grazing angles
MILLER, 1994
8
Our method
accessibility shading
rendering at grazing angles
MILLER, 1994
9
Related work

• Triangulation only
• CURLESS, 1997 (review)
• DAVIS et al., 2003, ZHANG et al., 2003
  (space-time stereo)
• Photometry only
• HORN, 1970 (shape from shading)
• WOODHAM, 1980 (photometric stereo)
• Hybrid approaches
• BERNARDINI et al., 2002 (normal map over
  triangulation)
• TERZOPOULOS, 1988 (optimize both together)

10
Outline of talk

• Error analysis
• Normal correction
• Position optimization
• Range images
• Arbitrary meshes
• Conclusions

11
Outline of talk

• Error analysis
• Normal correction
• Position optimization
• Range images
• Arbitrary meshes
• Conclusions

12
Test scanner
13
Visual error analysis

• Low frequency errors are hard to see
• Obtain reference data with hi-res scanner
• Color-code quantitative errors

position error
normal field error
color code
14
Stereo triangulation errors
triangulated positions
triangulation normals
60o
0o
30o
1mm
0
0.5
15
Photometric stereo errors
integrated positions
photometric normals
60o
0o
30o
1mm
0
0.5
16
Stereo triangulation errors
triangulated positions
triangulation normals
60o
0o
30o
1mm
0
0.5
17
Photometric stereo errors
integrated positions
photometric normals
60o
0o
30o
1mm
0
0.5
18
Summary and strategy

• Triangulated positions
• High-frequency noise
• Excellent low-frequency
• Photometric normals
• Low-frequency bias
• Excellent high-frequency

• Combine frequency bands
• Two-step process
• Use triangulated positions to correct photometric
  normals
• Use corrected normals to optimize triangulated
  positions

19
Outline of talk

• Error analysis
• Normal correction
• Position optimization
• Range images
• Arbitrary meshes
• Conclusions

20
Correcting Normals
Photometricnormals
Triangulatedpositions
Computenormals
Blur
Blur
21
Photometric stereo errors
integrated positions
photometric normals
60o
0o
30o
1mm
0
0.5
22
Corrected normal errors
integrated positions
corrected normals
60o
0o
30o
1mm
0
0.5
23
Outline of talk

• Error analysis
• Normal correction
• Position optimization
• Range images
• Arbitrary meshes
• Conclusions

24
Position optimization

• Use energy minimization
• Formulate normal and position constraints
• Use only linear error terms
• Reduces to linear least squares problem

25
Range images as input

• Position error

• Normal error

26
Corrected normal errors
integrated positions
corrected normals
60o
0o
30o
1mm
0
0.5
27
Optimized position errors
optimized positions
optimized normals
60o
0o
30o
1mm
0
0.5
28
Stereo triangulation results
triangulated positions
29
Our method
triangulated positions
optimized positions
480k v, 1M t, 91 CG, 25s
30
Alignment comparison
31
Arbitrary meshes as input

• Position error

• Normal error

32
Arbitrary mesh results
Input provided by Tim Weyrich, ETH Zurich and MERL
630k vertices, 1.3M triangles
33
Filling holes
Input provided by Andrew Jones et al. from USC
triangulated positions
34
Filling holes
Input provided by Andrew Jones et al. from USC
holes
35
Filling holes
Input provided by Andrew Jones et al. from USC
smoothed
170k v, 350k t
proxy geometry
36
Filling holes
Input provided by Andrew Jones et al. from USC
1 round
170k v, 350k t
45 s
37
Filling holes
Input provided by Andrew Jones et al. from USC
6 rounds
170k v, 350k t
4.5 min
38
Filling holes
Input provided by Andrew Jones et al. from USC
11 rounds
170k v, 350k t
8.25 min
39
Filling holes
Input provided by Andrew Jones et al. from USC
16 rounds
170k v, 350k t
12 min
40
Filling holes
Input provided by Andrew Jones et al. from USC
new geometry
Performance Geometry Capture for Spatially
Varying Relighting Relighting sketch Thursday,
1030 - 1215
41
Conclusions

• We presented a method that
• Combines normal and position measurements
• Produces results that are better than either
  alone
• Uses an efficient linear formulation
• Supports range images and arbitrary meshes
• Future work
• Use to fill holes (already did)
• Frame as surface evolution, solve with level sets

42
Acknowledgements

• Princeton GFX group for their help
• ETH Zurich, MERL, and USC for data
• National Science Foundation grants CCF-0347427
  and CCF-0446916 for funding

43
Thank you for your attention