Contextbased Surface Completion

1
Context-based Surface Completion
Andrei Sharf, Marc Alexa, Daniel Cohen-Or
2
Introduction

• Holes in models
• Imperfect range scanned data
• Complex objects with non-visible regions.
• Misalignment of multiple-views depth image scans.
• Material reflections.
• Surface editing operations.

3
Motivation

• Smooth filling is sufficient
• Small holes.
• Smooth surfaces.
• Smooth filling is insufficient
• Surfaces containing fine geometric detail.
• Topology of hole is more complex than a disk.

4
Context-based Completion

• Complete the missing region with patches that
  conform with its context

Smooth
Context-based
5
Method

• Import patches with matching context from the
  surface itself
• Analyze surface characteristics.
• Find best matching patch.
• Fit imported patch to boundary.

6
Related Work

• Example-based image completion
• Drori et al. 2003 Jia and tang 2003 Sun et
  al. 2003
• Texture synthesis
• Efros and Leung 1999 Efros and Freeman
  2001Wei and Levoy 2000 Ying et al. 2001
• Smooth surface completion
• Curless and Levoy 1996 Davis et al. 2002 Ilic
  and Fua 2003 Verdera et al. 2003 Liepa 2003
• Model-based surface reconstruction Savchenko et
  al. 2002
• Curve analogies Hertzmann et al. 2002

Drori et al. 03
Wei and Levoi 00
Liepa 03
Hertzmann et al. 02
7
Moving to 3D Surfaces

• Images have a regular spatial structure domain.
• Problems in 3D
• Topology and geometry of missing region.
• Fitting a patch to the boundary of the missing
  region.
• Definition of similarity of shapes.
• Definition of a surface patch.

8
Algorithm

• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
• Compute a local shape representation.
• Compute a shape signature.
• For each empty cell
• Find matching nonempty cell ?.
• Copy patch of ? into ?.
• Subdivide cells and repeat
• Until completed region matches its neighborhood.

9
Algorithm

• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
• Compute a local shape representation.
• Compute a shape signature.
• For each empty cell
• Find matching nonempty cell ?.
• Copy patch of ? into ?.
• Subdivide cells and repeat
• Until completed region matches its neighborhood.

10
Shape Representation

• Shape sampled point-set
• Range scan output
• Easy to merge
• Octree hierarchy on top of the point set
• Implicit surface approximation by fitting
  polynomials Ohtake et al. 2003
• General quadric
• Bivariate quadratic
• Edge or corner fitting

11
Shape Analysis

• A local signature of a cell consists
• Implicit shape characteristics
• Signed distance
• Normal variation
• Detail amount
• Depth in octree

12
Algorithm

• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
• Compute a local shape representation.
• Compute a shape signature.
• For each empty cell
• Find matching nonempty cell ?.
• Copy patch of ? into ?.
• Subdivide cells and repeat
• Until completed region matches its neighborhood.

13
Missing Region

• Automatic identification
• Intersect shape approximation with cells.
• Empty cell that intersects surface is part of
  missing region.
• Empty cell definition can alter due to refinement
  of missing region.

?
?
14
Matching
Find most similar non-empty cell

• Distance metrics
• dc Signature distance inside cells.
• da Signature distance of adjacent cells.
• dl Difference in amount of detail.

15
Matching-Candidate Set

• Non-empty cells of the same size
• Symmetry rotations of cells(?/2, ?, ...)
• Rotation of all point-set with some angle (?/4,
  ?/3, ...)

16
Transferring Points

• ICP
• Copy points into empty cell.
• Find closest point correspondence
• Align points rigidly
• Align points non-rigidly using polynomial form.

17
Algorithm

• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
• Compute a local shape representation.
• Compute a shape signature.
• For each empty cell
• Find matching nonempty cell ?.
• Copy patch of ? into ?.
• Subdivide cells and repeat
• Until completed region matches its neighborhood.

18
Davids Hair
Original
Down-sampled
Smooth completion
Context-based completion
19
Completion Process
Original
Final result
Initial approximation
20
Manual Editing of Bunny Model
21
Manual Editing of Knot Model
22
Scan of Youth Statue (Rear View)
Original
Smooth
Result
23
Scan of Human Bone
Original
Smooth
Result
24
Limitations Semantics
25
Sample Rate vs. Detail Frequency
26
Summary

• A fully automatic method to complete a missing
  region in a surface from its context.
• Completed patches geometrically conform with
  neighborhood.
• Incremental scale-space framework for finer
  approximation of the unknown region.

27
Future Work

• Explore with other spatial hierarchies.
• Enlarge the search space of examples by building
  a class based example set.
• Couple method with image completion methods for
  texture completion.
• Rotation and translation invariant signature

28

• Acknowledgments
• David Levin and Olga Sorkine
• Israel science foundation Israeli ministry of
  science
• Digital Michelangelo project 3D model repository
  Stanford 3D scanning repository Darmstadt
  university of technology Imager computer
  graphics laboratory of the university of British
  Columbia

29
Thank you