Image-Based Visual Hulls

1
Image-Based Visual Hulls

• Paper by Wojciech Matusik, Chris Buehler, Ramesh
  Raskar,Steven J. Gortler and Leonard McMillan
• http//graphics.lcs.mit.edu/wojciech/vh/
• Vortrag von Simon Dellenbach
• GDV Fachseminar 2001

2
Overview (1)

• Motivation
• Basics
• Viewpoint Model
• Visual Hull
• Epipolar Geometry
• Creating Image-Based Visual Hulls

3
Overview (2)

• Rendering IBVH
• System Implementation
• Summary Results
• Future Work
• Personal Opinion

4
Motivation (1)

• Traditional computer graphics, rendering..
• static synthetic scenes (CG Images)
• dynamic synthetic scenes (CG Animations)
• static acquired scenes (Image-Based Rendering)
• Acquire and render dynamic scenes in real-time
• appropriate representation
• rendering system

5
Motivation (2)
6
Viewpoint Model - Basics (1)
7
Visual Hull - Basics (2)

• Geometric shape obtained using silhouettes of
  object seen from number of views
• extruded silhouette cone-like volume limiting
  the extent of object
• intersection of volumes results in a visual hull
• more views ? better approximation of object
• limitation concavities cant be captured(e.g.
  an open box looks like a solid cube)

8
Visual Hull - Basics (3)
9
Epipolar Geometry - Basics (4)

• The tree points COP1,COP2,P form an epipolar
  plane
• Intersection of this plane with image planes
  results in epipolar lines
• The line connecting the two centers of projection
  COP1,COP2 intersects the image planes at the
  conjugate points e1 and e2 which are called
  epipoles

10
Epipolar Geometry - Basics (5)
11
Creating Image-Based Visual Hulls (1)

• Algorithm input
• set of k silhouettes (binary images) with
  associated viewpoints
• desired viewpoint (in this case, constructed
  visual hull is viewpoint-dependent)
• Algorithm output
• sampled image of the visual hull, each pixel
  containing a list of occupied intervals of space

12
Creating Image-Based Visual Hulls (2)

• The Basic Algorithm
• cast ray into space for each pixel in the desired
  view of the visual hull
• intersect ray with the k silhouette cones? k
  lists of intervals intersect together? single
  list of intersections of the viewing ray with the
  visual hull

13
Creating Image-Based Visual Hulls (3)
14
Creating Image-Based Visual Hulls (4)

• Trick due to Epipolar Geometry interval
  calculation can be done in image space of
  reference images
• 3D intersecting silhouette cone with viewing ray
• 2D intersecting projected viewing ray with
  silhouette

15
Creating Image-Based Visual Hulls (5)
16
Rendering IBVH (1)

• Reference images are used as textures
• For each pixel
• rank reference-image texture from best to
  worst according to angle, take reference with
  lowest
• avoid texturing surface points with an image
  whose line-of-sight is blocked by some other
  point of the visual hull
• consider visibility during shading based on
  visual hull (not actual geometry)

17
Rendering IBVH (2)
18
Rendering IBVH (3)
19
System Implementation (1)

• Four calibrated and triggered digital cameras
• One desktop PC per camera for capturing and
  pre-processing video frames (image segmentation)
• Silhouette and texture information sent to
  central server for IBVH processing

20
System Implementation (2)

• Server runs IBVH intersection and shading
  algorithms
• IBVH objects can be combined with OpenGL
  background
• System runs in real time with heavy
  optimization (like caching strategies for
  silhouette intersection)

21
System Implementation (3)
22
Summary Results

• Use visual hull as object shape approximation
• Using silhouette information from reference views
  to generate view dependent visual hull
• Reference images are used as textures
• Results
• Videoclips

23
Future Work

• Find Techniques for blending between textures to
  produce smoother transitions
• Scale up system by using larger number of cameras
• Split workload on multiple servers, as algorithm
  parallelizes fairly much
• Speed up viewing ray silhouette intersections
  (most expensive part of the computation)

24
Personal Opinion (1)

• Pros
• simple technique / low-cost hardware
• image-based representation partially compensates
  simplification problems
• epipolar geometry reduces 3D-intersection
  problems to 2D-intersections

25
Personal Opinion (2)

• Cons
• texture flipping during viewpoint transitions
  produces ugly results
• shadows are considered as part of the object
• preprocessing is really expensive(85 ms for
  image foreground segmentation)

26
The End
?
?
?

• If there are no questions,there wont be any
  answers.

?
?
?