Light Propagation Volumes in CryEngine 3

1
Light Propagation Volumes in CryEngine 3 Anton
Kaplanyan
Advances in Real-Time Rendering in 3D Graphics
and Games
AntonK_at_Crytek.de
2
Agenda

• Introduction
• CryEngine 3 lighting pipeline overview
• Core idea
• Applications (with video)
• Improvements
• Combination with other technologies (with video)
• Optimizations for consoles
• Conclusion and future work
• Live demo

3
Introduction into real-time graphics

• Strictly fixed budget per frame
• Many techniques are not physically-based
• Consistent performance
• Game production is complicated
• This talk is mostly about massive and indirect
  lighting
• This is a high level talk
• More implementation details in the paper

4
CryEngine 3 renderer overview (1 / 5)

• Xbox 360 / PlayStation 3 / DirectX 9.0c / 10 /
  (11 soon)

5
CryEngine 3 renderer overview (2 / 5)

• Unified shadow maps solution Mittring07

6
CryEngine 3 renderer overview (3 / 5)

• SSAO Kajalin09, Mittring09

7
CryEngine 3 renderer overview (4 / 5)

• Deferred lighting Mittring09
• Minimal G-Buffer
• Sun / Omni / Projectors / Caustics / Deferred
  light probes

8
CryEngine 3 renderer overview (5 / 5)

• Lighting accumulation pipeline
• Apply global / local hemispherical ambient
• Optionally Replace it with Deferred Light Probes
  locally
• Global illumination solution should take place
  here
• Multiply indirect term by SSAO to apply ambient
  occlusion
• Apply Direct Lighting on top of Indirect Lighting

9
Real-time rendering development trends

• Rendering is a multi-dimensional query
  Mittring09
• R R(View, Geometry, Material, Lighting)
• Divide-and-conquer strategy, some examples
• Shadow maps (decouple visibility queries)
• Deferred techniques (decouple lighting / shading)
• Screen-space techniques (SSAO, SSGI, etc.)
• Reprojection techniques (partially decouples
  view)
• Why?
• Less interdependencies gt more consistent
  performance
• Future trends parallel and distributed
  computations friendly

10
Paper reference icon

• This icon means that details are in the paper

TM
11
Light Propagation Volumes
12
Light Propagation Volumes Goals

• Decouples lighting complexity from screen
  coverage (resolutionoverdraw)
• Radiance caching and storing technique
• Massive lighting with point light sources
• Global illumination
• Participating media rendering (still work in
  progress)
• Consoles friendly (Xbox 360, PlayStation 3)

13
Related work

• Irradiance Volumes GSHG97, Tatarchuk04,
  Oat05
• Signed Distance Fields Evans06
• Lightcuts A Scalable Approach to Illumination
  WFABDG05
• Multiresolution Splatting for Indirect
  Illumination NW09
• Hierarchical Image-Space Radiosity for
  Interactive Global Illumination NSW09
• Non-interleaved Deferred Shading of Interleaved
  Sample Patterns SIMP06

14
SH Irradiance volumes

• A grid of irradiance samples is taken
  throughout the scene
• Each irradiance samplestored in SH form
• At render time, the volume is queried and
  near-by irradiance samples are interpolated to
  estimate the global illumination at a point in
  the scene

From GSHG97, Tatarchuk04
15
Low-frequency radiance volumes

• Similar to SH Irradiance Volumes Tatarchuk04
• Stores radiance distribution instead
• Low resolution 3D texture on GPU (up to 323
  texels)
• SH approximation is low order (up to linear band)
• Radiance is not smooth GSHG97
• But what is the error introduced by approximating
  it?

From GSHG97
16
Radiance approximation

• Error of the spatial approximation depends on
• density and size / radii of light sources
• Error of the angular approximation depends on
• Shape of light source
• Frequency of angular radiance distribution of
  light source
• Distance to the light source
• Compensated by the energy fall-off
• Preserves mean energy andmajor radiance flow
  direction
• Enough if we want to eventually get irradiance

17
Light propagation in radiance volume

• Start with given initial radiance distribution
  from emitters
• Iterative process of radiance propagation
• 6-points axial stencil for adjacent cells
• Gathering, more efficient for GPUs
• Energy conserving
• Each iteration adds to result, then propagates
  further

18
Light propagation in radiance volume
19
Rendering with Light Propagation Volume

• Regular shading, similar to SH Irradiance Volumes
• Simple 3D texture look-up using world-space
  position
• Integrate with normals cosine lobe to get
  irradiance
• Simple computation in the shader for 2nd order SH
• Lighting for transparent objects and
  participating media
• Deferred shading / lighting
• Draw volumes shape into accumulation buffer
• Supports almost all deferred optimizations

20
Massive Lighting with point light sources
21
Massive lighting

• Option 1 Inject initial energy, then propagate
  radiance
• A bit faster for crazy amount of lights
• Option 2 Add pre-propagated radiance into each
  cell
• Simple analytical equation in the shader for
  point lights
• Higher quality, no propagation error
• Error depends on the ratio (light source radius /
  cell size)
• Radius threshold for lighting with radiance
  volume

22
Glossy reflections with Light Propagation Volumes

• Accumulative traversal (ignores reflection
  occlusion)
• Several look-ups along reflected ray from camera
• Collect incoming radiance from this direction
• Integrate over the cone of incoming direction
• Cone angle depends on
• Glossiness of surface
• Distance from look-up to point p
• Approximates the integration with Phong BRDF

23
Glossy reflections example
24
Massive lighting Results

• NVIDIA GeForce GTX 280 GPU, Intel Core 2 Quad CPU
  _at_ 2.66 GHz, DirectX 9.0c API, HDR rendering _at_
  1280x720, no MSAA, Volume size 323

25
Massive lighting video
26
Global Illumination with Light Propagation Volumes
27
Global Illumination with Light Propagation Volumes

• Instant Radiosity Keller97
• The main idea is to represent light bouncing as a
  set of secondary light sources Virtual Point
  Lights (VPL)
• Splatting Indirect Illumination DS07
• Based on Instant Radiosity
• Reflective Shadow Maps (RSM) are used to generate
  initial set of VPLs on GPU
• Importance sampling of VPLs from RSM

28
Reflective Shadow Maps

• Reflective Shadow Map efficient VPL generator
• Shadow map with MRT layout depth, normal and
  color

29
Global Illumination with Light Propagation Volumes

• Inject the initial radiance from VPLs into
  radiance volume
• Point rendering
• Place each point into appropriate cell
• Using vertex texture fetch / R2VB
• Approximate initial radiance of each VPL with SH
• Simple analytical expression in shader
• Propagate the radiance
• Render scene with propagated radiance

30
Implementation details

• Light Propagation Volume moves with camera
• 3D cell-size snapping for volume movement
• 2D texel-size snapping for RSM movement
• RSM is higher in resolution than radiance volume
• Smart down-sampling of RSM

31
Global Illumination with Light Propagation Volumes
32
Issue Cell-alignment of VPLs

• Injection of VPLs involvesposition shifting
• Position of injected VLP becomes grid-aligned
• Consequence of spatial radiance approximation
• Unwanted radiance bleeding
• Lighting of double-sided and thin geometry

33
Cell-alignment of VPLs Bleeding example
34
Cell-alignment of VPLs Solution

• VPL half-cell shifting
• towards normal
• towards light direction
• Coupled by anisotropic bilateral filtering
• During final rendering pass
• Sample radiance with offset by surface normal
• Compute radiance gradient
• Compare radiance with radiance gradient

35
Cascaded Light Propagation Volumes for GI

• One grid is limited in dimensions and low
  resolution
• Multiresolution approach for radiance volumes
• Similar to Cascaded Shadow Maps technique SD02
• Preserves surrounding radiance outside of the
  view
• Each cascade is independent
• With separate RSM for each cascade
• Transmit radiance across adjacent edges
• Filter objects by size for particular RSM
• Efficient hierarchical representation of radiance
  emitters

36
Global Illumination Video
37
Global Illumination Combination with SSAO

• No secondary occlusion for light propagation
  volumes
• Can be approximated by Ambient Occlusion term

SSAO on, GI off
SSAO off, GI on
GI SSAO
38
Global Illumination Combination with SSGI

• Screen-Space Global Illumination RGS09
• Limitations of SSGI
• Only screen-space information
• Huge kernel radius for close objects
• Limitations of Light Propagation Volumes
• Local solution
• Low resolution spatial approximation
• Supplementing each other
• Custom blending

39
Global Illumination Combination with SSGI
SSGI on
SSGI off
40
Optimizations for consoles Xbox 360 / PS3

• 3D texture look-up with trilinear filtering
• Radiance volume is 32 bpp for all three SH
  textures
• Xbox 360, 3,5 ms per frame
• Vertex texture fetching for RSM injection
• Work-around to resolve into particular slice of
  3D texture
• PlayStation 3, 3,4 ms per frame
• Emulate signed blending in the shader
• R2VB for RSM injection (using memory remapping)
• Render to unwrapped 2D RT then remap as 3D
  texture

41
Future work

• Better radiance approximation
• Participating media rendering
• Occlusion for indirect lighting
• Multiple bounces
• Improve quality
• Improved propagation scheme
• Better angular approximation
• Adaptive grids
• Support for arbitrary types of light sources

42
References

• DS07 Dachsbacher, C., Stamminger, M. 2007.
  Splatting Indirect Illumination
• Evans06 Evans, A. 2006. Fast Approximations for
  Global Illumination on Dynamic Scenes
• GSHG97 Greger, G., Shirley, P., Hubbard, P.,
  Greenberg, D. 1997. The Irradiance Volume
• Isidoro05 Isidoro J. 2005. Filtering Cubemaps
  Angular Extent Filtering and Edge Seam Fixup
  Methods
• Kajalin09 Kajalin, V. 2009. Screen-space
  ambient occlusion, Shader X7
• Keller97 Keller, A. 1997. Instant radiosity
• Mittring07 Mittring, M. 2007. Finding Next Gen
  CryEngine 2
• Mittring09 Mittring, M. 2009. A bit more
  Deferred CryEngine3.
• NSW09 Nichols, G., Shopf, J., Wyman, C. 2009.
  Hierarchical Image-Space Radiosity for
  Interactive Global Illumination
• NW09 Nichols, G., Wyman, C. 2009.
  Multiresolution Splatting for Indirect
  Illumination
• Oat05 Oat, C., 2006 Irradiance Volumes for
  Real-Time Rendering, ShaderX 5
• RGS09 Ritschel, T., Grosch, T., Seidel, H.-P.
  2009. Approximating Dynamic Global Illumination
  in Image Space
• SD02 Stamminger, M., Drettakis, G. 2008.
  Perspective shadow maps
• SIMP06 Segovia, B., Iehl, J. C., Mitanchey, R.,
  Peroche, B. 2006. Non-interleaved Deferred
  Shading of Interleaved Sample Patterns
• Tatarchuk04 Tatarchuk, N. 2004. Irradiance
  Volumes for Games
• WFABDG05 Walter, B., Fernandez, S., Arbree, A.,
  Balda, K., Donkikian, M., Greenberg, D. 2005.
  Lightcuts A Scalable Approach to Illumination
• More details in the paper at http//www.crytek.com
  /technology/presentations/

43
Acknowledgment

• Michael Endres, Felix Dodd, Marco Siegel, Frank
  Meinl, Alexandra Cicorschi, Helder Pinto, Efgeni
  Bischoff and other artists and designers at
  Crytek for created scenes
• Martin Mittring, Vladimir Kajalin, Tiago Sousa,
  Ury Zhilinsky, Mark Atkinson, Evgeny Adamenkov
  and the whole Crytek RD team
• Special thanks to Carsten Dachsbacher and Natalia
  Tatarchuk

44
(No Transcript)
45
Thank you for your attention!Questions?
AntonK_at_Crytek.de