Tom%20Haber,%20Christian%20Fuchs,%20Philippe%20Bekaert,%20Hans-Peter%20Seidel,%20Michael%20Goesele,%20Hendrik%20Lensch

1
Relighting Objects from Image Collections

• Tom Haber, Christian Fuchs, Philippe Bekaert,
  Hans-Peter Seidel, Michael Goesele, Hendrik Lensch

2
Goal
3
Requirements
4
Geometry Pipeline
5
Related Work

• Georghiades
• Photometric stereo
• Point lights
• Simple reflectance model
• Yu et al.
• Spherical harmonics
• Static illumination
• Gaussian filtered mirror BRDF model

6
Overview
Input images
Bilinear model
Geometry
7
Rendering equation
R( )
8
Wavelets

• Haar wavelets
• O(N log N)
• Triple Product Wavelet Integrals for
  All-Frequency Relighting, Ng et al
• Octahedron Parameterization

9
Robustness

• Problem is Ill-posed!!
• Environment map
• Smoothness
• Iteratively reweighting least-squares vs. noise
• BRDFs
• More lighting conditions ? more stable
• Linear model with basis BRDFs
• Clustering (two-level approach)

10
BRDF representation

11
BRDF representation
Linear Model
12
BRDF basis functions

• BRDF database Matusik03
• Sum of basis functions
• Radial basis functions Zickler05
• Zernike polynomials
• Sum of Gaussians
• Parametric models
• Lafortune
• Phong
• Anisotropic

13
Synthetic Experiments
14
Validation Minerva dataset
15
Minerva Dataset
16
Results Minerva
17
Static Illumination
18
Static Illumination (2)

• Viewpoint not in image collection

19
Internet Dataset
20
Results Lady Liberty
21
Results Lady Liberty
22
Results Spheres
23
Venus Di Milo
24
Conclusion

• No assumptions placed on image set
• Meaningful separation possible
• Quality influenced by geometry precision
• Ambiguity between illumination and surface color

25
Thanks
We would like to thank all the flickr users for
contributing their photographs. The first
author was funded by tUL impulsfinanciering.
Part of the research at EDM is funded by the
IBBT. This work has been partially funded by the
DFG Emmy Noether fellowships (Le 1341/1-1, GO
1752/3-1) and the Max Planck Center for Visual
Computing and Communication (BMBF-KZ01IMC01).
26
Wavelets
27
Triple product (math)
Triple Product Wavelet Integrals for
All-Frequency Relighting, Ren Ng, Ravi
Ramamoorthi, Pat Hanrahan, SIGGRAPH 2004
28
Wavelet Rotation

29
Wavelet Rotation
30
Interpolation
31
Number of coefficients?
Basis Choice Number Non-Zero
Box O (N )
Pixels O (N)
Sph. Harmonics O (N 5 / 2)
Haar Wavelets O (N log N)
32
Representation

• Wavelet
• Multi-resolution
• Sparse
• Negative values
• Box filters
• Multi-resolution
• sparse
• Lower compressibility
• Spherical Harmonics
• Low frequency content
• Expensive

33
Spherical Parameterizations

• Probe
• Cube map
• Longitude Latitude
• Octahedron Praun03

Uffizi probe courtesy of Paul Debevec
34
Haar Tripling Coefficient Theorem

• The integral of 3 Haar wavelets is non-zero iff
• All three are the scaling function
• All three are co-square and different
• Two are identical, and the third overlaps at a
  coarser level