AllFrequency Precomputed Radiance Transfer using Spherical Radial Basis Functions and Clustered Tens

1
All-Frequency Precomputed Radiance Transfer using
Spherical Radial Basis Functions and Clustered
Tensor Approximation

• Yu-Ting Tsai and Prof. Zen-Chung Shih
• Department of Computer Science
• National Chiao Tung University, Taiwan

2
Precomputed Radiance Transfer (PRT)
From Sloan et al. 2003
3
Precomputed Radiance Transfer (PRT)
From Sloan et al. 2003
4
Precomputed Radiance Transfer (PRT)
From Sloan et al. 2003
5
Precomputed Radiance Transfer (PRT)

• Rendering Equation for Distant Illumination

Light Transfer Function
Spherical Harmonics
6
Motivation

• Problems
• Enormous size of all-frequency PRT data sets
• Memory bandwidth issues for PRT rendering using
  GPUs
• Challenges
• All-frequency PRT for glossy objects at real-time
  rates
• High-level data representation for PRT
• Compact storage space of all-frequency PRT data
  sets

7
Motivation

• Our Solutions
• Compact data representation spherical radial
  basis functions (SRBFs)
• Sophisticated compression technique clustered
  tensor approximation (CTA)

8
Related Work

• PRT for Static Scenes

Frequency
Our Target
High Frequency
Ng et al. 03
Liu et al. 04
Low Frequency
Sloan et al. 02
Sloan et al. 03
Rendering Performance
Real-Time
Interactive
9
Related Work

• Special Basis Functions
• Spherical harmonics (SH)
• Analogy of the Fourier series over the unit
  sphere
• Ortho-normal basis functions
• Difficult to model all-frequency signals
• Wavelets
• Multi-resolution analysis
• Localized both in spatial and frequency domain
• Typically fixed analysis pattern

10
Related Work

• Special Basis Functions
• Radial basis functions (RBFs)
• Related to sparse data interpolation and
  extrapolation
• Adaptive analysis pattern by adjusting parameters
• High computational costs
• Spherical radial basis functions (SRBFs)
• A new era of data presentation for radiance
  functions
• Modeling radiance functions in intrinsic domain
• Similar properties to RBFs
• May be promising for many problems of computer
  graphics

11
Related Work

• Dimensionality Reduction
• Linear model (PCA)
• Linear dependence among data
• Non-linear model (non-linear PCA, kernel PCA, and
  generalized PCA)
• Widely-adopted in other fields (computer vision,
  pattern recognition, etc.)
• Multi-linear model (tensor approximation)
• A popular approach in recent years

12
System Diagram
Off-Line
SRBF PRT Computation
Clustered Tensor Approximation
Scattered SRBF Approximation
Run-Time
SRBF Rotation
PRT Rendering
13
Spherical Radial Basis Functions

• Rotation-invariant and axis-symmetric functions
  on the unit sphere
• PRT data sets and lighting environments can be
  represented in intrinsic domain
• Adaptive to spatial variation by adjusting
  centers and bandwidth parameters
• Spherical integrals can be efficiently computed

Gaussian SRBFs with different bandwidth
parameters 2D Plot
A Gaussian SRBF 3D plot
14
Spherical Radial Basis Functions

• General Formula
• Spherical Integral

?
?
?
Geodesic distance between two points
Legendre Polynomial
15
Spherical Radial Basis Functions

• Represent A Spherical Function with SRBFs
• Abel-Poisson SRBF
• Gaussian SRBF

SRBF coefficient
SRBF Center
SRBF Bandwidth
16
System Diagram
Off-Line
SRBF PRT Computation
Clustered Tensor Approximation
Scattered SRBF Approximation
Run-Time
SRBF Rotation
PRT Rendering
17
SRBF Representation for PRT

• BRDF Factorization

Light-Dependent BRDF
View-Dependent BRDF
18
SRBF Representation for PRT
Combined with Light-Dependent BRDF
Approximate with SRBFs
Blocked
Transfer Matrix
Sampled Visibility Data
19
System Diagram
Off-Line
SRBF PRT Computation
Clustered Tensor Approximation
Scattered SRBF Approximation
Run-Time
SRBF Rotation
PRT Rendering
20
Clustered Tensor Approximation


Vertex Basis Matrix
Transfer Matrix
Light Basis Matrix
View Basis Matrix
Core Tensor
Cluster C
21
System Diagram
Off-Line
SRBF PRT Computation
Clustered Tensor Approximation
Scattered SRBF Approximation
Run-Time
SRBF Rotation
PRT Rendering
22
Scattered SRBF Approximation

• An Optimization Problem
• Non-linear optimization
• Minimizing squared errors
• Three sets of parameters
• SRBF coefficients
• SRBF centers
• SRBF bandwidth
• Highly-coupled parameters
• Dependent on initial guess

23
Scattered SRBF Approximation

• An Iterative Approach
• Heuristic initial guess
• Optimize only one set of parameters in each stage
• Three optimization stages
• SRBF centers non-linear optimization
• SRBF bandwidth non-linear optimization
• SRBF coefficients ordinary least-squares (OLS)
  or regularized least-squares (RLS) projection
• Iterate until convergence

24
Scattered SRBF Approximation
Reference Image 6 32 32
SH SE 94.81 Sloan et al. 03
Wavelets SE 8.80 Ng et al. 03
Our Approach SE 0.11
25
Results
Liu et al. 04 (13 fps)
Our Approach (51 fps)
26
Results
Reference Image (lt 0.1 fps)
Our Approach (74 fps)
27
Results
Reference Image (lt 0.1 fps)
Our Approach (48 fps)
28
Various Configurations of CTA
Larger Vertex Basis
Larger Light Basis
29
Discussion

• Limitations of current SRBF PRT system
• No all-frequency mirroring effects
• Two-pass rendering
• More approximate than wavelet approaches (uniform
  set of SRBFs for each vertex)
• Due to computational costs, not actual limitation
  of SRBFs
• May be solved in the future
• No fully dynamic changes of lighting environments
  at run-time
• Rotation only

30
Discussion

• Limitations of current CTA algorithm
• Clustering along only one dimension
• Clustering along more than two dimensions?
• CTA for dense data sets
• Sparse data sets?
• Linear dependence among data
• Non-linear model?

31
Conclusions

• Spherical radial basis functions are efficient in
  representing PRT data sets
• Our CTA algorithm is promising for compressing
  all-frequency PRT data sets
• Our scattered SRBF approximation algorithm is
  effective in modeling HDR lighting environments

32
Future Work

• Improvements of our PRT system
• Integrate full global illumination solutions
• Improve the accuracy of compressed PRT data
• Scattered SRBF PRT representation
• Dynamic PRT
• Level-of-detail PRT

33
Future Work

• Sophisticated and robust techniques for
  approximating large data sets with SRBFs
• Heuristic approaches
• Statistical models
• Applications of SRBFs to other fields
• Approximating measured BRDF data with SRBFs
• Importance sampling of spherical functions

34
Future Work

• Non-linear extension of CPCA and CTA algorithms
• PRT signals are highly non-linear
• Lighting environments, BRDF, and visibility
• Non-linear model for CPCA and CTA?
• Non-linear dependence among PRT data sets
• More compact size of compressed data

35
Future Work

• Applications of CTA algorithm to other fields
• Bidirectional texture functions
• Image-based rendering on GPUs
• Analysis of high-dimensional visual data sets
• Analysis of time-dependent visual data sets

36
Thank You for Attention