Ambient Occlusion and Edge Cueing for enhancing Real Time Molecular Visualization

1
Ambient Occlusion and Edge Cueing for enhancing
Real Time Molecular Visualization

• Marco Tarini, Paolo Cignoni, Claudio Montani

2
BackgroundMolecular Graphics

• The discipline and philosophy of studying
  molecules and their properties through graphical
  representations
• Stereotypical Scientific Visualization task
• 3D shape ? Molecule functionality
• Long story
• Long bibliography
• Many established techniques and principles
• Capstone after lunch by Jane Richardson!

Hemagglutinin image courtesy of Riumplus
image courtesy of Accelrys
3
BackgroundMolecular Graphics
images (animations, stereo images...)
3D molecular structure
3D represen- tation(s)
modelling
rendering
insight, understanding

• Many 3D Metaphors
• Ball and Sticks
• "Licorice" modes
• Space Fill (CPK)
• Solvent Excluded Surfaces (Connolly's)
• Various kind of isosurfaces
• (electron density, potential...)
• Higher level abstractions
• e.g. ribbons

4
BackgroundMolecular Graphics
images (animations, stereo images...)
3D molecular structure
3D represen- tation(s)
modelling
rendering
insight, understanding

• Problems
• Readability
• Unfamiliar, awkward 3D structures
• Hard to understand 3D shapes (spatial
  relationships, etc)
• Efficiency
• Geometric complexity (need for real time)(e.g.
  for very large proteins)

5
BackgroundMolecular Graphics
images (animations, stereo images...)
3D molecular structure
3D represen- tation(s)
modelling
rendering
insight, understanding

• The content of this talk
• Advanced rendering techniques can be of great
  help here!
• Advanced rendering techniques can be adapted to
  the specific case of molecular rendering

6
The rest of this talkad-hoc Advanced Rendering
Techniques

• Impostors for spheres, cylinders
• Ambient occlusion (for impostors)
• Depth aware borders (for impostors)
• Localized depth cueing effect (for impostors)

7
Impostor based rendering

• For each primitive (sphere or cylinder)
• render a screen-oriented impostor quad
• per vertex
• displace vertex in screen space
• preprocess all common sub-expressions
• for each fragment
• test membership (discard outliers fragments)
• compute
• z
• normal
• lighting
• texture coords (as we will see)

8
The rest of this talkad-hoc Advanced Rendering
Techniques

• Impostors for spheres, cylinders
• Ambient occlusion (for impostors)
• Depth aware borders (for impostors)
• Localized depth cueing effect (for impostors)

9
Ambient Occlusion

• Global lighting technique
• Approximation of global illumination
• Concept
• light each point p with normal np with the
  irradiance I(p) (quantity of light directly
  reaching p from any direction)

• in our case, L is k (a constant) if p visible
  from ?, 0 if p shadowed
• Effect
• lighted by daylight under a cloudy sky

10
Ambient Occlusionwhy

• Known to conceive realism (but, we dont care)

Local lighting
Ambient Occlusion
11
Ambient Occlusionwhy

• Known to conceive 3D shape understanding
• perceptual study in Langer Bulthoff, TechRep
  99
• Especially in our case (molecules)
• large scale structures composed by multitudes of
  tiny shapes ? normals noise
• see also particle systems by Bigler etAl.
  VCGT06

12
Ambient Occlusionwhy
Local lighting
Ambient Occlusion
13
Ambient Occlusionhow

• Store computed Irradiance of each point on
  molecule surface
• (fast precomputation)
• use it during rendering
• PROBLEM where do we store it?
• use a texture
• for impostors!

14
Texturing the molecule
x,y,z object space
u,v texture space
F
E
x,y,z gnomicprojection
unfolding
A
B
D
C
G
H
Praun and Hoppe. TG03
y,z gnomicprojection
unfolding
15
Texturing the molecule
x,y,z object space
u,v texture space
F
E
A
B
D
C
G
H
16
Texturing the molecule
x,y,z object space
u,v texture space
F
E
A
B
D
C
G
H
texel replications
bilinear filetering
17
Texturing the molecule

• Global Texture for the molecule
• one texture patch for each
• atom (sphere) or
• bond (cylinder)
• in the scene
• pack together all texture patches
• (trivial)

v
ATOM 1
BOND 1
ATOM 2
ATOM 3
ATOM 4
BOND 2
ATOM 5
ATOM 6
ATOM 7
BOND 2
BOND 4
BOND 5
BOND 6
BOND 7
ATOM 8
BOND 8
ATOM 9
ATOM 10
ATOM 11
ATOM 12
BOND 9
BOND 10
BOND 11
(unused)
u
18
Texturing the molecule

• For example
• small molecule
• 80 patches
• 40 atoms
• 40 bonds
• texture size 512x512 texels
• patch size 50x50 texels
• large molecule
• 120.000 patches
• 60.000 atoms
• 60.000 bonds
• texture size 2048x2048 texels
• patch size 6x6 texels

v
IRRADIANCE MAP
u
19
Computing irradiance map on the GPU

• Get N directions w1..N sampling the unit sphere
• For each i in (1..N)
• render new depth-map from view direction wi
• render over irradiance-map for each patch
  for each fragment (texel) f
• invert (u,v) map get pos p, normal np
• project p over depth-map check visibility
  (compare depths)
• if visibile, accumulate light K(npwi) (if gt0)
  at f (with alpha blending)

20
Computing irradiance map on the GPU

• Get N directions w1..N sampling the unit sphere
• For each i in (1..N)
• render new depth-map from view direction wi
• render over irradiance-map for each patch
  for each fragment (texel) f
• invert (u,v) map get pos p, normal np
• project p over depth-map check visibility
  (compare depths)
• if visibile, accumulate light K(npwi) (if gt0)
  at f (with alpha blending)

• Render
• flat impostors!
• mid z (better result)
• no z-displacing (much faster)

irradiance-map
21
Computing irradiance map on the GPU

• Get N directions w1..N sampling the unit sphere
• For each i in (1..N)
• render 2 new depth-map from view directions wi
  -wi
• render over irradiance-map for each patch
  for each fragment (texel) f
• invert (u,v) map get pos p, normal np
• project p over depth-map check visibility
  (compare depths)
• if visibile, accumulate light K(npwi) (if gt0)
  at f (with alpha blending)

depth-map -wi
depth-map wi
irradiance-map
22
The rest of this talkad-hoc Advanced Rendering
Techniques

• Impostors for spheres, cylinders
• Ambient occlusion (for impostors)
• Depth aware borders (for impostors)
• Localized depth cueing effect (for impostors)

23
Depth aware borders

• Example
• See demo

24
Depth aware bordershow

• Single pass technique!

depth
25
Depth aware bordershow

• Single pass technique!

B
A
C
depth
26
Depth aware bordershow

• Single pass technique!

B
A
C
depth
27
Depth aware bordershow

• Single pass technique!

B
A
C
depth
28
The rest of this talkad-hoc Advanced Rendering
Techniques

• Impostors for spheres, cylinders
• Ambient occlusion (for impostors)
• Depth aware borders (for impostors)
• Localized depth cueing effect (for impostors)

29
Localized depth cueing (halos)

• Depth cueing (fog)
• common technique to visually separate far from
  near
• BUT
• far parts lose contrast (global loss of contrast)
• cannot discriminate between too many planes
• Our alternative localized depth cueing
  (halos)
• like fog, but concentrate effects near depth
  jumps!
• Similar to T.Luft, C.Colditz, O.Deussen.
  SIGGRAPH 2006 (parallel result)
• See Demo!

Picasso
Matisse
Seuret
Dalì
30
Localized depth cueing (halos)

• First pass
• render geometry and depth buffer
• Second pass (alpha blended)
• render coronas around each atom
• choose alpha according to
• screen distance to atom
• distance of current fragment depth from
  destination depth

31
Putting all together

• Combining
• Impostors for spheres, cylinders.
• Ambient occlusion (for impostors)
• Depth aware borders (for impostors)
• Localized depth cueing effect (for impostors)
• with standard techniques
• Direct light (Lambertian Phong)
• Cast shadow
• Standard depth cueing (fog)
• Toon shading (for illustrative renderings)
• Antialiasing (render to 2x, resample to screen)

32
Implementation

• OpenSource project
• win installer
• unix, mac versions coming up
• reads PDB files
• exports hi res. images (e.g. for print
  publications)
• and small animations (e.g. for web sites,
  presentations)
• Just Google for

GIF animation