Advanced Computer Graphics Spring 2006

1
Advanced Computer Graphics
(Spring 2006)

• COMS 4162, Lecture 21 Image-Based Rendering
• Ravi Ramamoorthi

http//www.cs.columbia.edu/cs4162
2
To Do / Motivation

• Work hard on assignment 4
• This last series of lectures covers (at a high
  level) some more advanced topics and areas of
  current research interest in modern rendering

3
Course Outline

• 3D Graphics Pipeline

4
Next few slides courtesy Paul Debevec SIGGRAPH
99 course notes
5
(No Transcript)
6
(No Transcript)
7
Image-Based Modeling and Rendering

• Generate new views of a scene directly from
  existing views
• Pure IBR (such as lightfields) no geometric
  model of scene
• Other IBR techniques try to obtain higher quality
  with less storage by building a model

8
IBR Pros and Cons

• Advantages
• Easy to capture images photorealistic by
  definition
• Simple, universal representation
• Often bypass geometry estimation?
• Independent of scene complexity?
• Disadvantages
• WYSIWYG but also WYSIAYG
• Explosion of data as flexibility increased
• Often discards intrinsic structure of model?

9
(No Transcript)
10
(No Transcript)
11
IBR A brief history

• Texture maps, bump maps, env. maps 70s
• Poggio et al. MIT Faces, image-based
  analysis/synthesis
• Modern Era
• Chen and Williams 93, View Interpolation Images
  with depth
• Chen 95 Quicktime VR Images from many
  viewpoints
• McMillan and Bishop 95 Plenoptic Modeling Images
  w disparity
• Gortler et al, Levoy and Hanrahan 96 Light Fields
  4D
• Shade et al. 98 Layered Depth Images 2.5D
• Debevec et al. 00 Reflectance Field 4D
• Inverse rendering methods (Sato,Yu,Marschner,Boivi
  n,)
• Fundamentally, sampled representations in
  graphics

12
Outline

• Overview of IBR
• Basic approaches
• Image Warping
• Light Fields
• Survey of some recent work

13
Warping slides courtesy Leonard McMillan
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Outline

• Overview of IBR
• Basic approaches
• Image Warping
• 2D depth. Requires correspondence/disparity
• Light Fields 4D
• Survey of some recent work

19
(No Transcript)
20
Plenoptic Function

• L(x,y,z,q,f,t,l)
• Captures all light flow in a scene
• to/from any point (x,y,z),
• in any direction (q,f),
• at any time (t),
• at any frequency (l)
• Enough information toconstruct any imageof the
  scene at any time

Funkhouser
21
Plenoptic Function Simplifications

• Represent color as RGB eliminate l
• Static scenes ignore dependence on t
• 7D ? 3 ? 5D

22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Lumigraph Postprocessing

• Obtain rough geometric model
• Chroma keying (blue screen) to extract
  silhouettes
• Octree-based space carving
• Resample images to two-plane parameterization

32
Lumigraph Rendering

• Use rough depth information to improve rendering
  quality

33
Lumigraph Rendering

• Use rough depth information to improve rendering
  quality

34
Lumigraph Rendering
35
Unstructured Lumigraph Rendering

• Further enhancement of lumigraphsdo not use
  two-plane parameterization
• Store original pictures no resampling
• Hand-held camera, moved around an environment

36
Unstructured Lumigraph Rendering

• To reconstruct views, assign penalty to each
  original ray
• Distance to desired ray, usingapproximate
  geometry
• Resolution
• Feather near edges of image
• Construct camera blending field
• Render using texture mapping

37
Unstructured Lumigraph Rendering
Blending field
Rendering
38
Outline

• Overview of IBR
• Basic approaches
• Image Warping
• 2D depth. Requires correspondence/disparity
• Light Fields 4D
• Survey of some recent work

39
LDIs

• Layered depth images Shade et al. 98

Slide from Agrawala, Ramamoorthi, Heirich, Moll,
SIGGRAPH 2000
40
LDIs

• Layered depth images Shade et al. 98

LDI
41
LDIs

• Layered depth images Shade et al. 98

LDI
(Depth, Color)
42
(No Transcript)
43
Surface Light Fields

• Miller 98, Nishino 99, Wood 00
• Reflected light field (lumisphere) on surface
• Explicit geometry as against light fields.
  Easier compress

44
Acquiring Reflectance Field of Human Face
Debevec et al. SIGGRAPH 00

• Illuminate subject from many incident directions

45
Example Images
Images from Debevec et al. 00
46
Conclusion (my views)

• Real issue is compactness/flexibility vs.
  rendering speed
• IBR is use of sampled representations. Easy to
  interpolate, fast to render. If samples images,
  easy to acquire.
• IBR in pure form not really practical
• WYSIAYG
• Explosion as increase dimensions (8D transfer
  function)
• Ultimately, compression, flexibility needs
  geometry/materials
• But lots of recent work (some in course) begins
  to correct these issues
• Right question is tradeoff compactness/efficiency
• Factored representations
• Understand sampling rates and reconstruction