A Theory of Locally Low Dimensional Light Transport - PowerPoint PPT Presentation

1 / 67
About This Presentation
Title:

A Theory of Locally Low Dimensional Light Transport

Description:

Convolution formula for glossy reflections and shadows. Ramamoorthi and Hanrahan 01 ... Function of slope of. dimensionality curve ... – PowerPoint PPT presentation

Number of Views:26
Avg rating:3.0/5.0
Slides: 68
Provided by: www1CsC
Category:

less

Transcript and Presenter's Notes

Title: A Theory of Locally Low Dimensional Light Transport


1
A Theory of Locally Low Dimensional Light
Transport
  • Dhruv Mahajan
    (Columbia University)
  • Ira Kemelmacher-Shlizerman
    (Weizmann Institute)
  • Ravi Ramamoorthi
    (Columbia University)
  • Peter Belhumeur
    (Columbia University)

2
Image Relighting
Ng et al 2003
3
Relighting Linear Combination
Nimeroff et al 94
Hallinan 94
Dorsey 95
Lighting Intensities

Images lit by directional light sources
4
Relighting Matrix Vector Multiply
B
T
L


Output Image
Input Lighting
Vector
(Unfolded Cubemap)
Transport Matrix
5
Light Transport Computational Cost
  • Light transport matrix dimensions
  • 512 x 512 images
  • 6 x 32 x 32 6144 cubemap lighting
  • Multiplication / Relighting cost
  • Approximately 1010 computations per frame
  • Multiplication intractable in real time
  • Need to compress the light transport

6
Light Transport SVD
T
T
U
S
V
V
L
Lighting
Eigenvalues
Transport Matrix
Vector
Relit Image
Projection Weights
Hallinan 94
Basis Images
7
Light Transport SVD
T
V
Eigenvalues
Transport Matrix
- Global Dimensionality
Large
8
Global Dimensionality
  • Computation still intractable

9
Locally Low Dimensional Light Transport
p rows
SVD
Transport Matrix
p pixels
Dimensionality of the patch
Locally Low Dimensional Transport
Lighting Resolution
10
Previous Work
  • Blockwise PCA Nayar et al. 04
  • Image divided in to fixed size square patches
  • Each patch compressed using PCA
  • Clustered PCA Sloan et al. 03
  • Object divided in to fixed number of clusters
  • Each cluster compressed using PCA

11
Previous Work
  • Surface light fields
  • Nishino et al. 01
  • Chen et al. 02
  • General reflectance fields
  • Matusik et al. 02
  • Garg et al. 06
  • Compression
  • JPEG, MPEG

No Theoretical Analysis
Dimensionality vs Patch Size?
Dimensionality vs Material Properties?
Dimensionality vs Global Effects ?
12
Local Light Transport Dimensionality
  • Analysis of local light transport dimensionality

Dimensionality
P
1
13
Local Light Transport Dimensionality
  • Analysis of local light transport dimensionality

Dimensionality
2 x 2
14
Local Light Transport Dimensionality
  • Analysis of local light transport dimensionality

Dimensionality
15
Local Light Transport Dimensionality
  • Analysis of local light transport dimensionality

Dimensionality
16
Local Light Transport Dimensionality
  • Analysis of local light transport dimensionality

Dimensionality
17
Local Light Transport Dimensionality
  • Analysis of local light transport dimensionality

Dimensionality
18
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
Dimensionality
19
Overhead Cost
Overhead Cost Projection Weights
Dimensionality cost number of bases
20
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
P
21
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
22
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
23
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
24
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
25
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
26
Rendering Cost
  • Theoretical analysis of rendering cost

Overhead cost for rendering
Rendering cost Dimensionality Overhead
Optimal
Patch Size
27
Contributions
  • Analysis of dimensionality of local light
    transport
  • Change of dimensionality with size
  • Diffuse and glossy reflections
  • Shadows
  • Analyzing rendering cost
  • Analytical formula for optimal patch size
  • Practical Applications
  • Fine tuning parameters of existing methods
  • Scale images to very high resolutions
  • Develop adaptive clustering algorithm

28
Local Light Transport Dimensionality
  • Analysis of local light transport dimensionality

Dimensionality
29
Dimensionality vs. Patch Size
slope 1
Diffuse/Specular BRDF
Large Area
linear relationship
slope - rate of change of dimensionality
Independent of material properties
Dimensionality Patch Area
pixels
dimensionality
30
Dimensionality vs. Patch Size
slope lt 1
Diffuse/Specular BRDF
Small Area
sub - linear relationship
pixels
dimensionality
31
Mathematical Tools for Analysis
  • Convolution formula for glossy reflections and
    shadows
  • Ramamoorthi and Hanrahan 01
  • Basri and Jacobs 01
  • Ramamoorthi et al 04
  • Szegos Eigenvalue Distribution Theorem
  • Eigenvalues of the light transport matrix of the
    patch
  • Fourier Scale and Convolution Theorems
  • Dimensionality as a function of patch size

32
Central Result
Patch Dimensionality
Patch Dimensionality
Bandwidth of BRDF
Patch Area
Bandwidth of BRDF
Patch Area
Constant
Constant
33
Central Result
Patch Dimensionality
Bandwidth of BRDF
Patch Area
Constant
low frequency
high frequency
99 Energy
Bandwidth
BRDF/ Material Properties
34
Central Result
(
)
(
)
Patch Dimensionality
Bandwidth of BRDF
Patch Area
Bandwidth of BRDF
Patch Area
Constant
Constant
Large Area
Diffuse/Specular BRDF
35
Large Area
(
)
(
)
Patch Dimensionality
Bandwidth of BRDF
Patch Area
Diffuse/Specular BRDF
36
Large Area
(
)
(
)
Patch Dimensionality
Bandwidth of BRDF
Patch Area
)
(
)
(
(
)
Patch Dimensionality
Bandwidth of BRDF
Patch Area
Diffuse/Specular BRDF
37
Large Area
linear relationship
)
(
)
(
(
)
Patch Dimensionality
Bandwidth of BRDF
Patch Area
slope 1
Diffuse/Specular BRDF
38
Small Area
sublinear relationship
)
(
)
(
(
)
Patch Dimensionality
Bandwidth of BRDF
Patch Area
slope lt 1
Diffuse/Specular BRDF
39
Contributions
  • Analysis of dimensionality of local light
    transport
  • Change of dimensionality with size
  • Glossy reflections
  • Shadows
  • Analyzing rendering cost
  • Analytical formula for optimal patch size
  • Practical Applications
  • Fine tuning parameters of existing methods
  • Scale images to very high resolutions
  • Develop adaptive clustering algorithm

40
Visibility Function
Visibility Function 0
Lighting Directions
Blocker
Visibility Function 1
Visibility Function 1
P
41
Shadows
Light Transport Visibility Function
slope 1
slope .5
Diffuse and Specular BRDF
Shadows
  • Dimensionality changes slowly in presence of
    shadows

42
Shadows Step Blocker
Light Transport Visibility Function
Step Blocker
Lighting
Direction
Same Visibility Function
Different Visibility Function
Dimensionality changes only along one dimension
Dimensionality vPatch Area
log (Dimensionality) .5 log(Patch Area)
43
Shadows Step Blocker
Light Transport Visibility Function
Step Blocker
Same Visibility Function
Different Visibility Function
Dimensionality changes only along one dimension
Dimensionality vPatch Area
log (Dimensionality) .5 log(Patch Area)
44
Contributions
  • Analysis of dimensionality of local light
    transport
  • Change of dimensionality with size
  • Glossy reflections
  • Shadows
  • Analyzing rendering cost
  • Analytical formula for optimal patch size
  • Practical Applications
  • Fine tuning parameters of existing methods
  • Scale images to very high resolutions
  • Develop adaptive clustering algorithm

45
Local Light Transport Dimensionality
  • Analysis of dimensionality of local light
    transport
  • Diffuse and Glossy reflections, dimensionality
    area
  • Shadows, dimensionality varea

Bandwidth of BRDF
Patch Dimensionality
Patch Area
Constant
46
Contributions
  • Analysis of dimensionality of local light
    transport
  • Change of dimensionality with size
  • Glossy reflections
  • Shadows
  • Analyzing rendering cost
  • Analytical formula for optimal patch size
  • Practical Applications
  • Fine tuning parameters of existing methods
  • Scale images to very high resolutions
  • Develop adaptive clustering algorithm

47
Overhead Cost
Dimensionality
48
Overhead Cost
Overhead
Dimensionality
P
49
Overhead Cost
Overhead
Dimensionality
50
Overhead Cost
Overhead
Dimensionality
51
Overhead Cost
Overhead
Dimensionality
52
Overhead Cost
Overhead
Dimensionality
53
Overhead Cost
Overhead
Dimensionality
54
Rendering Cost
Overhead
Rendering Cost
Dimensionality
55
Rendering Cost vs. Patch Size
Large Patch size
gt
Rate of increase in dimensionality
Rate of decrease in overhead
Increasing patch size increases total cost
Overhead
Linear regime
Rendering Cost
Dimensionality
56
Rendering Cost vs. Patch Size
Large Patch size
gt
Rate of increase in dimensionality
Rate of decrease in overhead
Increasing patch size increases total cost
Linear regime
Rendering Cost
Dimensionality
Overhead
Rendering cost Dimensionality Overhead
57
Rendering Cost vs. Patch Size
Small Patch size
lt
Rate of increase in dimensionality
Rate of decrease in overhead
Increasing patch size decreases total cost
Overhead
Sublinear regime
Rendering Cost
Dimensionality
58
Rendering Cost vs. Patch Size
Small Patch size
lt
Rate of increase in dimensionality
Rate of decrease in overhead
Increasing patch size decreases total cost
Sublinear regime
Rendering Cost
Dimensionality
Overhead
Rendering cost Dimensionality Overhead
59
Rendering Cost vs. Patch Size
Intermediate size

Rate of increase in dimensionality
Rate of decrease in overhead
Total cost minimum
Overhead
Rendering Cost
Minimum
Dimensionality
60
Rendering Cost vs. Patch Size
Intermediate size

Rate of increase in dimensionality
Rate of decrease in overhead
Total cost minimum
Rendering Cost
Minimum
Dimensionality
Overhead
Rendering cost Dimensionality Overhead
61
Optimal Patch Size
Optimal Patch Size
- Global Dimensionality
62
Optimal Patch Size
Optimal Patch Size
- Global Dimensionality
- Function of slope of dimensionality curve
- From our theoretical analysis
- Empirically from the given dataset
Dimensionality Curve
63
Optimal Patch Size CPCA Example
- Global Dimensionality
Optimal Patch Size
- Function of slope of
dimensionality curve
Total cost
110 220 330 440
550
average cluster size
Face dataset across lighting
64
Glossy Reflections
- Global Dimensionality
Optimal Patch Size
- Function of slope of
dimensionality curve
Independent of material properties
Number of pixels in the patch increases with
glossiness
65
Contributions
  • Analysis of dimensionality of local light
    transport
  • Change of dimensionality with size
  • Glossy reflections
  • Shadows
  • Analyzing rendering cost
  • Analytical formula for optimal patch size
  • Practical Applications
  • Fine tuning parameters of existing methods
  • Scale images to very high resolutions
  • Develop adaptive clustering algorithm

66
Setting Optimal Patch Size CPCA
67
Setting Optimal Patch Size CPCA
cost per pixel
clusters
220
114.78
Estimated
114.78-130
130-600
large
310.7
11
24000 vertices
6 X 32 X 32
45.0 Hz.
Cube Map
68
Contributions
  • Analysis of dimensionality of local light
    transport
  • Change of dimensionality with size
  • Glossy reflections
  • Shadows
  • Analyzing rendering cost
  • Analytical formula for optimal patch size
  • Practical Applications
  • Fine tuning parameters of existing methods
  • Scale images to very high resolutions
  • Develop adaptive clustering algorithm

69
Scaling of Cost With Resolution
- Global Dimensionality
- Function of slope of dimensionality curve
new resolution
Independent of patch resolution
Optimal patch size same for both resolutions
Subdivide More
70
Scaling of Cost With Resolution
Sub-linear increase in cost with resolution
Increase in resolution -
Increase in cost -
new resolution
1.85
71
Scaling of Cost With Resolution
  • Sublinear increase in cost with resolution

800 x 600
1024 1024
72
Scaling of Cost With Resolution
73
Summary
  • Analysis of dimensionality of local light
    transport
  • Diffuse and Glossy reflections, dimensionality
    area
  • Shadows, dimensionality varea
  • Analysis of rendering cost
  • Optimal patch size
  • Scaling of cost with resolution
  • Practical Applications
  • Setting optimal parameters in existing methods
  • Adaptive clustering algorithms

74
Summary
  • Analysis of dimensionality of local light
    transport
  • Diffuse and Glossy reflections, dimensionality
    area
  • Shadows, dimensionality varea
  • Analysis of rendering cost
  • Optimal patch size
  • Scaling of cost with resolution
  • Practical Applications
  • Setting optimal parameters in existing methods
  • Adaptive clustering algorithms

Patch Dimensionality
Bandwidth of BRDF
Patch Area
Constant
75
Summary
  • Analysis of dimensionality of local light
    transport
  • Change of dimensionality with size
  • Glossy reflections, dimensionality area
  • Shadows, dimensionality varea
  • Analyzing rendering cost
  • Derive optimal patch size
  • Practical Applications
  • Fine tuning parameters of existing methods
  • Scale to very high resolutions
  • Develop adaptive clustering algorithms

76
Future Work
  • More solid theoretical foundation
  • High dimensional appearance compression
  • Representation
  • ECCV 2006, PAMI 2007
  • Analysis of light transport in frequency domain
  • TOG, Jan. 2007
  • Analysis of light transport in gradient domain
  • Siggraph 2007
  • Analysis of general local light transport for
    patches
Write a Comment
User Comments (0)
About PowerShow.com