Data Representation and Approximation Techniques for RealTime Visual Data Rendering

1
Data Representation and Approximation Techniques
for Real-Time Visual Data Rendering
Student Yu-Ting Tsai Advisor Prof. Zen-Chung
Shih Department of Computer Science National
Chiao Tung University, Taiwan
2
Outline

• Introduction
• Data Representation
• Spherical Radial Basis Functions
• Approximation Techniques
• Generalized Multi-Way Analysis
• SRBF-Based Spherical Wavelets
• Permuted PCA
• Unstructured K-ary Wavelets
• Conclusion

3
Introduction
4
Visual Data Rendering

• Analytic Models
• Render visual effects from formulae, procedures,
  or algorithms
• Data-Driven Models
• Render visual effects from data
• Data sources
• Measured data
• Image-based data
• Precomputed data

5
Visual Data Rendering

• Challenges
• Data size
• Real-time performance
• Scalability
• Edibility
• Meaningful factors
• Special data sets

6
Previous Proposals

• Novel Data Representation
• Spherical Radial Basis Functions (SRBFs)
• Sophisticated Approximation Technique
• Clustered Tensor Approximation (CTA)

Vertex
Transfer Matrix
Light
View
7
Data Representation
8
Related Work

• Data Representation
• Primitive elements
• Points, lines, polygons, voxels, , etc.

9
Related Work

• Data Representation
• Topological structures
• Meshes, graphs, hierarchical structures, , etc.

10
Related Work

• Data Representation
• Parametric models
• Splines, parametric surfaces, analytic models, ,
  etc.

Lambertian Reflectance Model
11
Related Work

• Data Representation
• Functional models
• Fourier series, harmonic functions, wavelets,
  radial basis functions, , etc.

Green, GDC 03
12
Related Work

• Data Representation
• Functional models
• Spherical harmonics Ramamoorthi and Hanrahan,
  SIGGRAPH 01 Sloan et al., SIGGRAPH 02
• Wavelets Matusik et al., SIGGRAPH 03 Ng et
  al., SIGGRAPH 03, 04
• Radial basis functions Carr et al., SIGGRAPH
  01 Turk and OBrien, ACM TOG 02

SRBFs
13
Spherical Radial Basis Functions

• Simple Example

Center
Coefficient
Bandwidth
14
Spherical Radial Basis Functions

• SRBF Representation

15
Spherical Radial Basis Functions

• SRBF Representation

16
Spherical Radial Basis Functions

• SRBF Representation

17
Spherical Radial Basis Functions

• SRBF Representation

18
Spherical Radial Basis Functions

• SRBF Representation

19
Spherical Radial Basis Functions

• SRBF Representation

20
Spherical Radial Basis Functions

• SRBF Representation

21
Spherical Radial Basis Functions

• SRBF Representation

22
Spherical Radial Basis Functions

• Rotation-invariant and axis-symmetric functions
• Radiance functions can be represented in
  intrinsic domain
• Adaptive to spatial variation by adjusting
  centers and bandwidth parameters
• Spherical integrals can be efficiently computed

Gaussian SRBFs with different bandwidth
parameters 2D Plot
A Gaussian SRBF 3D plot
23
Applications

• Measured BRDFs

Weng, Master Thesis 06
24
Applications

• Importance Sampling

Weng, Master Thesis 06
25
Applications

• Image-Based Lighting
• SRBFs behave as filtering kernels

Larger Bandwidth
26
Applications

• HDR Environment Approximation
• Alternating least-squares optimization
• Precondition / initial guess
• Multi-resolution optimization
• Color space (LAB)
• Separate analysis
• Sub-block, sub-band, sub-energy
• Simplification
• Statistical methods
• Bayesian, HMM, MLS

Reference Image
Our Approach
27
Applications

• More Applications
• Dynamic PRT
• Image-based data sets
• Normal / tangent / bump map estimation
• Any radiance / spherical functions

28
Multi-Variate SRBFs

• Anisotropic SRBFs
• Bandwidth changes with azimuthal angle around
  center
• More degrees of freedom
• Contour function
• Constructed from multiple uni-variate SRBFs
• Spherical integrals?

29
Approximation Techniques
30
Related Work

• Basis Pursuit
• Clustering, vector quantization, Fourier series,
  wavelets, radial basis functions
• Matrix Factorization
• Singular value decomposition, homo-morphic
  factorization, non-negative factorization
• Dimensionality Reduction
• Principal components analysis
• Clustered PCA, locally linear embedding, iso-map,
  Laplacian eigen-map
• Multi-way analysis, sub-space analysis

31
Approximation Techniques

• Common Essence
• Transformations
• Basis functions
• Sparse solutions

Coherence!
32
Approximation Techniques

• Maximize Coherence
• Linear / non-linear transformations
• Orthogonal / non-orthogonal basis functions
• Local / global coherence
• Uniform / adaptive approximations
• Prior knowledge
• Latent variables

33
Approximation Techniques

• Generalized Multi-Way Analysis
• SRBF-Based Spherical Wavelets
• Permuted PCA
• Unstructured K-ary Wavelets

34
Generalized Multi-Way Analysis
35
Two-Way Analysis

• BRDF Matrix

Light
View
Sampled BRDF
36
Two-Way Analysis

• Principal Component Analysis

Light-Dependent
View-Dependent
Sampled BRDF
37
Two-Way Analysis

• Advantages
• Low computational costs
• Global optimum (least-squares errors)
• Efficient reconstruction
• Disadvantages
• Only for two-way data sets
• Global coherence
• Uniform approximations

38
Multi-Way Analysis
Light
View
39
Multi-Way Analysis
Texel

Texel Basis Matrix
Light
Light Basis Matrix
View Basis Matrix
Core Tensor
View
40
Clustered Multi-Way Analysis
Texel


Light
Cluster C
View
41
Clustered Multi-Way Analysis
Iterative Process

Texel Basis Matrix
Tensor Approximation
Light Basis Matrix
View Basis Matrix
Core Tensor
Cluster C
42
Multi-Way Analysis

• Advantages
• For high-dimensional data sets
• High compression ratio
• Coherence exploitation in each sub-space
• Efficient reconstruction
• Disadvantages
• High computational costs
• Local optimum

43
Multi-Way Analysis

• Major Drawbacks
• Alternating least-squares
• Local optimum
• Best rank of each sub-space is unknown
• Linear analysis in each sub-space
• Clustering along only one dimension

44
Generalized Multi-Way Analysis
Texel

Texel Basis Matrix
Light
Light Basis Matrix
View Basis Matrix
Core Tensor
View
45
Generalized Multi-Way Analysis

• No independence assumptions
• Alternating vs. simultaneous analysis
• Correlation model
• Simultaneously derive all the basis matrices

46
Generalized Multi-Way Analysis

• Adaptive Shrinkage
• Simple, intuitive, and practical
• Correlation model
• Closed-form solutions?

47
Generalized Multi-Way Analysis

• Generalization of CTA
• Non-linear extension
• Clustering along more than one dimension
• Destruction before construction

48
Generalized Multi-Way Analysis

49
Generalized Multi-Way Analysis

50
Applications
Light
View
51
Applications
Raw BTFs (192x192x120x270) 1.1 GB
Compressed BTFs (96x96x32x32) 18.1 MB (Multi-way
factorization) SE 0.61
52
Applications

• All-Frequency Bi-Scale PRT

53
Applications

• All-Frequency Bi-Scale PRT

54
Applications

• More Applications
• Time-varying appearance analysis
• Image-based rendering on GPUs
• All-frequency PRT
• More complex surface appearance
• View-dependent bump and displacement maps
• Time-varying appearance
• Real-time dynamic all-frequency PRT

55
Conclusion
56
Conclusion

• RBF-based methods can be easily extended to SRBFs
• Generalized multi-way analysis aims at overcoming
  major drawbacks of previous multi-way analysis
• Destruct the structure before construction to
  maximize coherence

57
The End
Thank You Questions?