Is There Anything Comparable to Spherical Harmonics But Simpler

1
Is There Anything Comparable to Spherical
Harmonics But Simpler?

• Tien-Tsin Wong The Chinese University of Hong
  Kong
• Chi-Sing Leung City University of Hong Kong

2
Motivation

• SH lighting offers fast realistic rendering
• But its mathematics is

Quite complicated!
3
What This Talk is About

• Advanced lecture with equations
• Spherical Radial Basis Function
  (SRBF)Comparable, but intuitive and simple
• How it works
• How to use SRBF lighting in games
• How to reduce data size safely
• Results

4
SH vs SRBF (Round 1)

• Lets first show how SRBF performs

5
SH vs SRBF (Round 2)

• more

6
SH vs SRBF (3)

• SRBF is comparable to SH in quality
• So, what is its advantage?
• Simpler mathematics
• More efficient local illumination
• Fast projection of distant environment (can even
  be time-varying)

7
Precomputed Lighting

• Basic idea
• Pre-computating the lighting effect
• Compressing it offline
• Decompressing it during the runtime
• Rendering as linear combination

8
Basis Functions

• Key is to select a good basis
• For SH, all basis functions are different

• Inefficient to evaluate the functions
• Local illumination is inefficient

9
Simpler Basis?

• Any simpler basis? or even homogeneous?
• The answer is spherical radial basis function
  (SRBF)
• A spherical function f is approximated by a
  linear combination of SRBFs

10
What is SRBF?

• All basis functions Ri have in same mathematical
  form and same shape

• Only different in orientation
• Intuitively, each basis function is responsible
  for a part of the sphere

11
What is SRBF? (2)

• In fact, Ri can be any symmetric function
• Gaussian
• Poisson

12
How SRBF Looks Like?

• On plane, a RBF is just a hill
• In spherical domain, a SRBF is a lobe

13
Things to Decide

• Number of basis functions
• SRBF centers
• SRBF coverage

14
Number of Basis Functions

• More SRBFs, better quality, more memory
• More SRBFs are needed if
• Hard shadow
• Specular highlight

original
15
SRBF Center

• How to orient the SRBF or position its SRBF
  center on sphere?
• Uniform distribution
• Possible sampling patterns
• icosahedron HEALPix

Check out our 20-min lecture at 1200pm tomorrow
in Room 44
16
SRBF Coverage

• Last thing to decide is the coverage (width) of
  SRBF

• The same coverage for all SRBFsHence, no extra
  storage overhead

17
SRBF Coverage (2)

• What if the coverage is too small?

18
SRBF Coverage (3)

• What if the coverage is too large?

19
SRBF Coverage (4)

• Appropriate coverage
• For each center, the geodesic distance to its
  closest neighbor is found
• Their average is used as the coverage

20
Rendering
21
Rendering

• Local illumination
• Local illumination such as point light sources
  are common in games
• A spotlight or moving a point source closer
  cannot be achieved with a distant environment
• Distant Environment
• Including distant area light sources
• Pseudo global illumination

22
Local Illumination

• Given a point or directional source, we get a
  light vector L for each surface element
• Use L to obtain the precomputed radiance or
  reflectance, i.e. f(L)
• Require fast evaluation of basis function Ri(L)
  to obtain f(L)

23
Local Illumination (2)

• For SRBF, evaluation of all basis functions is
  just a lookup to the same 1D table
• In contrast, all SH basis functions are different
  and inefficient (require time-consuming
  evaluation or many lookup tables)

This is the 1D table
24
Prepare for Rendering

• Note that each vertex holds a vector of SRBF
  coefficients ci
• In practice, we group the coefficients so that
  the i-th coefficients are grouped to form the
  i-th coefficient map

25
Prepare for Rendering (2)

• i.e.

• Why? Coeff. maps can be linearly combined by
  fragment shaders

26
Directional Source

• For illustration purpose, lets start with the
  simplest case, directional source

27
Result
VIDEO (dir.avi)
28
Point Source

• Each vertex p has its own light vector Lp
• Need to evaluate Ri multiple times for each vertex

29
Point Source (2)

• Attenuation can be added to the result of
  previous linear combination

30
Result
VIDEO (pt.avi)
31
Distant Environment

• To simulate global illumination, the scene can be
  illuminated by a distant environment E(L)
• Reflected radiance from surface element p

Represented by SRBF
Represented in dual space
32
Distant Environment (2)

• For SH, both E and can be stored as SH
• For SRBF, E is stored in dual space
• What is dual space?

A dot product Just like in SH
33
Fast Projection

• This dual space projection is computed only once
  for the environment map
• Graphically,

Multiplying each SRBF to the environment and
summing
34
Fast Projection (2)

• Why is it fast?
• Very low resolution maps are sufficient for such
  integration
• Ri(L) can be looked up
• E(L) can be downsampled in preprocess stage
• Such integration (dot product) can be computed by
  GPU in real-time
• Even for 770 coeff., 3072 samples on the maps are
  sufficient

X
35
Time-Varying Environment

• Dont be fooled by the integration
• The projection can be computed in real-time even
  the environment is time-varying
• A special case rotation of environment
• More general environment sequence

36
Result (Rotation)
VIDEO (distenv-rotation.avi)
37
Result (Environment Video)
VIDEO (distenv-timevary.avi)
38
More Details Maths
39
Glossy Surfaces

• View-dependent
• 4D transfer function (double sphere)
• Two approaches
• Double projection
• Decomposition

40
Double Projection

• Transfer functions at all vertices are aligned to
  a common (global) coordinate system
• First, fix V and project for L
• Then, project for V

41
Decomposition

• Decompose the transfer function f into
• gk(V), functions of V
• fk(L), functions of L
• By singular value decomposition,
• fk(L) can be SRBF projected as usual

42
Rotation of Basis

• Given a SRBF coeff. vector c, how to rotate f
  without re-projection?
• is the transformation
• This can be done by multiplying two matrices

43
Rotation of Basis (2)

• The elements in the above two matrices are
• Wow, that seems to be overkill
• Dont worry, all of them can be obtained by
  looking up another 1D table!

44
Rotation of Basis (3)

• Think about two SRBF, Ri and Rj
• The integration means the
  summation of the white region above
• For certain SRBF kernels (e.g. Poisson), the
  above integration has a simple close-form solution

45
Rotation of Basis (4)

• As SRBF is symmetric, the integration depends
  only on the angle between 2 SRBF centers
• We can construct a 1D table indexed by the dot
  product of 2 centers
• Much simpler and efficient than the analytical
  equations of SH rotation

46
Determine Coefficients

• How?
• Least square fit
• Step 1 projection
• Step 2 least square fit
• The same A-1 shown before

47
Noise Vulnerability

• The coefficients obtained are very noise
  sensitive
• If they are further compressed,

48
Noise-Proof Estimation

• Why?
• Because A is near singular
• magnitudes of coeff. are very large
• Most compression introduce following noise
• Larger magnitudes larger error

49
Noise-Proof Estimation (2)

• Solution control magnitudes during estimation
• Iteratively find out the smallest value of l
• such that

50
Noise-Proofing

• Here is the result

51
Discussions

• Truncating the basis function may result in
  holes because each SRBF is responsible for part
  of the sky
• Dilemma of constant coverage (single-scale SRBF)

52
Constant Coverage

• If f contains high frequency, and large coverage
  is selected

f
basis function
53
Constant Coverage (2)

• If f contains low frequency, and small coverage
  is selected

f
basis function
54
Multiscale SRBF

• Solution is to use multiscale SRBF
• Divides SRBF into groups (levels)
• Each group has different coverage (from small to
  large)

level 1
level 2
level 3
55
Comparison
single-scale(large width)
single-scale(small width)
56
Advantages

• True all-frequency rendering
• Multi-resolution analysis
• Continuous wavelet approach

57
Conclusion

• Visually, SRBF is comparable to SH
• Mathematically, SRBF is simpler than SH
• Computationally, SRBF is more efficient than SH
• So, why not use SRBF
• There are demo source codes on our websites

58
References

• Ease to read
• T. T. Wong, C. S. Leung and K. H. Choy, Lighting
  Precomputation Using the Relighting Map", Shader
  X3 Advanced Rendering with Direct X and OpenGL,
  Edited by W. Engel, Charles River Media, 2005,
  pp. 379-392.
• For mathematics
• C. S. Leung, T. T. Wong, P. M. Lam and K. H.
  Choy, An RBF-based Image Compression Method for
  Image-based Rendering, IEEE Transactions on
  Image Processing, Vol. 15, No. 4, April 2006, pp.
  1031-1041.
• Further questions
• Tien-Tsin Wong ttwong_at_cse.cuhk.edu.hk
• Chi-Sing Leung eeleungc_at_cityu.edu.hk

59
Web Resources

• Updated version of this presentation
• http//www.cse.cuhk.edu.hk/ttwong/papers/srbf/sr
  bf.html
• http//www.ee.cityu.edu.hk/csleung
• Sample source codes and tools
• http//www.cse.cuhk.edu.hk/ttwong/demo/srbf/srbf
  .html

60
Acknowledgments

• Ping-Man Lam and Gary Ho for implementation,
  figures, and video
• Liang Wan for preparation of figures
• This work is supported by Research Grants Council
  of the Hong Kong Special Administrative Region,
  under RGC Earmarked Grants (Project No.
  CUHK417005 and CityU115606)

61
Q A
62
The End
63
Time-Varying Environment

• Let E (L) be a distant environment
• gi projection
• Rotation a special case of time varying E(G, L)