RealTime High Quality Rendering - PowerPoint PPT Presentation

Loading...

PPT – RealTime High Quality Rendering PowerPoint presentation | free to download - id: 1ace81-MzdiN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

RealTime High Quality Rendering

Description:

RealTime High Quality Rendering – PowerPoint PPT presentation

Number of Views:52
Avg rating:3.0/5.0
Slides: 56
Provided by: ravirama8
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: RealTime High Quality Rendering


1
Real-Time High Quality Rendering
  • COMS 6160 Fall 2004, Lecture 5
  • Signal-Theoretic Representations of Appearance

http//www.cs.columbia.edu/ravir/6160
2
Research Projects
  • Mathematical foundations of appearance
  • High quality real-time rendering
  • Volumetric and Multiple Scattering Effects
  • Acquisition, Rendering with real lighting,
    materials
  • Complex lighting, materials in computer vision

3
Real-Time Rendering
  • Motivation Interactive rendering with
    natural illumination and realistic, measured
    materials

4
Image-Based Rendering
Original Photograph
5
Lighting-Insensitive Recognition
  • Illuminate subject from many incident directions
    Space of images as lighting is varied

6
Example Images
Images from Debevec et al. 00
7
Motivation and Approach
  • Many current applications in graphics, vision
    hard to solve directly
  • High dimensionality, sampling rate, computational
    cost
  • Signal-theoretic approach
  • Illumination and reflection functions are signals
  • Tools Sampling theory, frequency domain
    (spherical harmonics), convolution, wavelets
  • New theoretical results spherical harmonic
    convolution theorem, wavelet tripling coefficient
    analysis

8
Challenges
  • Illumination complexity
  • Material / view complexity
  • Transport complexity (shadows)
  • Putting it together

9
Environment Maps
Miller and Hoffman, 1984
Later, Greene 86, Cabral 87, 99,
10
Irradiance Environment Maps
Incident Radiance (Illumination Environment Map)
Irradiance Environment Map
11
Assumptions
  • Diffuse surfaces (relaxed later)
  • Distant illumination
  • No shadowing (relaxed later)
  • Hence, Irradiance is a function of surface normal

12
Computing Irradiance
  • Classically, hemispherical integral for each
    pixel
  • Lambertian surface is like low pass filter
  • Frequency-space analysis (spherical harmonics)

Incident Radiance
Irradiance
13
Analytic Irradiance Formula
  • Lambertian surface acts like low-pass filter

0
0
1
2
Basri Jacobs 01 Ramamoorthi Hanrahan 01a
14
9 Parameter Approximation
Order 2 9 terms
Exact image
0
RMS Error 1
1
For any illumination, average error lt 3 Basri
Jacobs 01
2
-1
-2
0
1
2
Ramamoorthi and Hanrahan 01b
15
Real-Time Rendering
  • Simple procedural rendering method (no textures)
  • Requires only matrix-vector multiply and
    dot-product
  • In software or NVIDIA vertex programming hardware
  • Widely used in Games (AMPED for Microsoft Xbox),
    Movies (Pixar, Framestore CFC, …)

16
Computer Vision Complex Illumination
  • Low Dimensional Subspace
  • Lighting-Insensitive Recognition (Basri and
    Jacobs 01, Lee et al. 01, Ramamoorthi 02, …)
  • Photometric stereo, shape acquisition

17
Convolution for general materials

Spherical harmonic analysis
Ramamoorthi and Hanrahan 01
18
Related Theoretical Work
  • Qualitative observation of reflection as
    convolution Miller Hoffman 84, Greene
    86, Cabral et al. 87,99
  • Reflection as frequency-space operator DZmura
    91
  • Lambertian reflection is convolution Basri
    Jacobs 01
  • Our Contributions
  • Explicitly derive frequency-space convolution
    formula
  • Formal quantitative analysis in general 3D case
  • Apply to real-time, inverse rendering, computer
    vision

19
Natural Lighting, Realistic Materials

Ramamoorthi and Hanrahan 02
20
Inverse Rendering
  • 3 photographs of cat sculpture
  • Complex unknown illumination
  • Geometry known
  • Estimate microfacet BRDF and distant lighting

21
New View, Lighting
Photograph
Rendering
Ramamoorthi and Hanrahan, 01c
22
Challenges
  • Illumination complexity
  • Material (BRDF)/view complexity
  • Transport complexity (shadows) Relighting
  • Putting it together

Ng, Ramamoorthi, Hanrahan 03
23
Existing Fast Shadow Techniques
We know how to render very hard, very soft shadows
Sloan et al. 2002, 03
Sen, Cammarano, Hanrahan, 2003
Shadows from smooth lighting (precomputed
radiance transfer)
Shadows from point-lights (shadow maps, volumes)
24
Assumptions
  • Static geometry
  • Precomputation
  • Real-Time Rendering (relight all-frequency
    effects)

25
Relighting as a Matrix-Vector Multiply
26
Relighting as a Matrix-Vector Multiply
Output Image (Pixel Vector)
  • Input Lighting (Cubemap Vector)

Transport Matrix
27
Problem Definition
  • Matrix is Enormous
  • 512 x 512 pixel images
  • 6 x 64 x 64 cubemap environments
  • Full matrix-vector multiplication is intractable
  • On the order of 1010 operations per frame
  • How to relight quickly?

28
Sparse Matrix-Vector Multiplication
Choose data representations with mostly
zeroes Vector Use non-linear wavelet
approximation on lighting Matrix
Wavelet-encode transport rows
29
Haar Wavelet Basis
30
Non-linear Wavelet Approximation
  • Wavelets provide dual space / frequency locality
  • Large wavelets capture low frequency area
    lighting
  • Small wavelets capture high frequency compact
    features
  • Non-linear Approximation
  • Use a dynamic set of approximating functions
    (depends on each frames lighting)
  • By contrast, linear approx. uses fixed set of
    basis functions (like 25 lowest frequency
    spherical harmonics)
  • We choose 10s - 100s from a basis of 24,576
    wavelets

31
Non-linear Wavelet Light Approximation
Wavelet Transform
32
Non-linear Wavelet Light Approximation
Non-linear Approximation
Retain 0.1 1 terms
33
Error in Lighting St Peters Basilica
Sph. Harmonics
Non-linear Wavelets
Relative L2 Error ()
Approximation Terms
Ng, Ramamoorthi, Hanrahan 03
34
Output Image Comparison
Top Linear Spherical Harmonic
Approximation Bottom Non-linear Wavelet
Approximation
35
Video
36
Challenges
  • Illumination complexity
  • Material (BRDF)/view complexity
  • Transport complexity (shadows) Relighting
  • Putting it together (relight and change view)

Ng, Ramamoorthi, Hanrahan 04
37
Changing Only The View
38
Problem Characterization
  • 6D Precomputation Space
  • Distant Lighting (2D)
  • View (2D)
  • Rigid Geometry (2D)
  • With 100 samples per dimension
  • 1012 samples total!! Intractable computation,
    rendering

39
Factorization Approach
6D Transport

1012 samples

4D Visibility
4D BRDF
108 samples
108 samples
40
Double Product Integral Relighting
41
Triple Product Integral Relighting
Changing Surface Position
Changing Camera Viewpoint
42
Triple Product Integral Relighting
  • 300,000 vertices
  • 6 x 64 x 64 cubemaps
  • 0.1 - 1 sparsity for visibility, BRDF
  • 3-5 seconds / frame

43
Relit Images (3-5 sec/frame)
44
Triple Product Integrals
45
Basis Requirements
  • Need few non-zero tripling coefficients
  • Need sparse basis coefficients

46
1. Number Non-Zero Tripling Coeffs
47
Basis Requirements
  • Need few non-zero tripling coefficients
  • Need sparse basis coefficients

48
2. Sparsity in Light Approx.
Pixels
Relative L2 Error ()
Wavelets
Approximation Terms
49
Summary of Basis Analysis
  • Choose (Haar) wavelet basis because
  • Few non-zero tripling coefficients
  • Sparse basis coefficients

50
Summary of Wavelet Results
  • Derive direct O(N log N) triple product algorithm
  • Dynamic programming can eliminate log N term
  • Final complexity linear in number of retained
    basis coefficients

51
Broader Computational Relevance
  • Clebsch-Gordan triple product series for
    spherical harmonics in quantum mechanics (but not
    focused on computation)
  • Essentially no previous work graphics, applied
    math
  • Same machinery applies to basic operation
    multiplication
  • Signal multiplication for audio, image
    compositing,….
  • Compressed signals/videos (e.g. wavelets JPEG
    2000)



52
Analytic Volume Scattering (fog, mist)
Original Image
Joint work with Narasimhan, Nayar (submitted to
TOG 04)
53
Monte Carlo Importance Sampling
54
Comparison 300 samples/pixel
Sampling Lafortune Fit
Our Method Factored rep.
Lawrence, Rusinkiewicz, Ramamoorthi 04
55
Summary
  • Many current applications in graphics, vision
    hard to solve directly
  • High dimensionality, sampling rate, computational
    cost
  • Understand computational structure of reflection
    and illumination
  • Decompose into lower-dimensional problems
  • Exploit coherence, low-complexity structures
  • Signal-processing approach
  • Find correct representations and computational
    tools for analyzing, representing, sampling
    lighting and appearance
  • New theoretical results Analytic convolution
    theorem in spherical harmonics, triple product
    analysis in wavelets
About PowerShow.com