Texture Mapping

1
Texture Mapping Other Fun Stuff
2
Last session?

• How Renderman works
• More on global illumination (Irradiance caching,
  bidirectional path tracing, Photon mapping)
• Computer Graphics Challenges in Games
• Recent CG research _at_ MIT

3
Modern Graphics Pipeline

• Perform projection of vertices
• Rasterize triangle find which pixels should be
  lit
• Compute per-pixel color
• Test visibility, update frame buffer

4
Ray Casting vs. Rendering Pipeline

• Ray Casting
• For each pixel
• For each object
• Send pixels to the scene
• Discretize first

• Rendering Pipeline
• For each triangle
• For each pixel
• Project scene to the pixels
• Discretize last

5
Ray Casting vs. Rendering Pipeline

• Ray Casting
• For each pixel
• For each object
• Ray-centric
• Needs to store scene in memory
• (Mostly) Random access to scene

• Rendering Pipeline
• For each triangle
• For each pixel
• triangle centric
• Needs to store image (and depth) into memory
• (Mostly) random access to frame buffer

Which is smaller? Scene or Frame? Frame Which is
easiest to access randomly? Frame because regular
sampling
6
Ray Casting vs. Rendering Pipeline

• Ray Casting
• For each pixel
• For each object
• Whole scene must be in memory
• needs spatial acceleration to be efficient
• Depth complexity no computation for hidden
  parts
• Atomic computation
• More general, more flexible
• Primitives, lighting effects, adaptive
  antialiasing

• Rendering Pipeline
• For each triangle
• For each pixel
• Harder to get global illumination
• Needs smarter techniques to address depth
  complexity (overdraw)
• Primitives processed one at a time
• Coherence geometric transforms for vertices
  only
• Good bandwidth/computation ratio
• Minimal state required, good memory behavior

7
Movies
both pipeline and ray tracing
8
Games
pipeline
9
Simulation
pipeline (painter for a long time)
10
CAD-CAM Design
pipeline during design, anything for final image
11
Architecture
ray-tracing, pipeline with preprocessing for
complex lighting
12
Virtual Reality
pipeline
13
Visualization
mostly pipeline, ray-tracing for high-quality eye
candy, interactive ray-tracing is starting
14
Medical Imaging
same as visualization
15
Question?
16
Photo-textures
17
Texture Interpolation

• Specify a texture coordinate (u,v) at each vertex
• Can we just linearly interpolate the values in
  screen space?

18
Interpolation - What Goes Wrong?

• Linear interpolation in screen space

texture source
what we get
what we want
19
Visualizing the Problem

• Notice that uniform steps on the image plane do
  not correspond to uniform steps along the edge.

20
Linear Interpolation
P(s)
p(t)

• On screen linear interpolation over t
• On triangle linear interpolation over s
• Our goal relate t to s, using the fact that
  p(t)p(s) is the projection of P(s)

21
Linear Interpolation
P(s)
p(t)
linear interpolation in screen space
after projection
interpolation in 3-space
p(s)
P(s)
22
Perspective Correct Interpolation
We need a mapping from t values to s values
Solve for s in terms of t
? Needs the z information (see previous lecture),
or the w1/z information
23
Modern graphics hardware

• Can interpolate many per-vertex values
• ? per fragment value
• Interpolation is perspective correct
• used for texture coordinates
• But also for normals, color, etc.

24
Questions?
25
Sampling Texture Maps

• How to map the texture area seen through the
  pixel window to a single pixel value?

textured surface(texture map)
image plane
circular pixel window
26
Sampling Texture Maps

• When texture mapping it is rare that the
  screen-space sampling density matches the
  sampling density of the texture.

64x64 pixels
Original Texture
Minification for Display
Magnification for Display
27
Linear Interpolation

• Tell OpenGL to use a tent filter instead of a box
  filter.
• Magnification looks better, but blurry
• (texture is under-sampled for this resolution)
• Oh well.

28
Questions?
29
Minification Examples of Aliasing
point sampling
30
Solution

• 1/ we can supersample the final image
• But only offsets the Nyquist limit
• 2/ we can blur the texture (prefilter)
• The right way to go
• But how much blur?

31
Spatial Filtering

• Remove the high frequencies which cause artifacts
  in texture minification.
• Compute a spatial integration over the extent of
  the pixel
• This is equivalent to convolving the texture with
  a filter kernel centered at the sample (i.e.,
  pixel center)!
• Expensive to do during rasterization, but an
  approximation it can be precomputed

projected texture in image plane
box filter in texture plane
32
MIP Mapping

• Construct a pyramid of images that are
  pre-filtered and re-sampled at 1/2, 1/4, 1/8,
  etc., of the original image's sampling
• During rasterization we compute the index of the
  decimated image that is sampled at a rate closest
  to the density of our desired sampling rate
• MIP stands for multum in parvo which means many
  in a small place

33
MIP Mapping Example

• Thin lines may become disconnected / disappear

MIP Mapped (Bi-Linear)
Nearest Neighbor
34
MIP Mapping Example

• Small details may "pop" in and out of view

MIP Mapped (Bi-Linear)
Nearest Neighbor
35
Examples of Aliasing

• Texture Errors

point sampling
mipmaps linear interpolation
36
Storing MIP Maps

• Can be stored compactly
• Illustrates the 1/3 overhead of maintaining the
  MIP map

37
Finding the mip level
area pre-filtered in MIP-map
textured surface(texture map)
image plane
circular pixel window

• Square MIP-map area is a bad approximation

38
Finding the MIP level
How does a screen-space change dt relates to a
texture-space change du,dv. This sounds like the
derivatives, ( du/dt, dv/dt ). They can be
computed simply using the chain rule
du, dv
dt
39
Finding the MIP level
What we'd like to find is the step size that a
uniform step in screen-space causes in
three-space, or, in other words how a
screen-space change relates to a 3-space change.
This sounds like the derivatives, ( du/dt, dv/dt
). They can be computed simply using the chain
rule
Notice that the term being squared under the
numerator is just the w plane equation that we
are already computing. The remaining terms are
constant for a given rasterization. Thus all we
need to do to compute the derivative is a square
the w accumulator and multiply it by a couple of
constants. Now, we know how a step in
screen-space relates to a step in 3-space. So how
do we translate this to an index into our MIP
table?
40
MIP Indices
Actually, you have a choice of ways to translate
this gradient value into a MIP level.
This also brings up one of the shortcomings of
MIP mapping. MIP mapping assumes that both the u
and v components of the texture index are
undergoing a uniform scaling, while in fact the
terms du/dt and dv/dt are relatively independent.
Thus, we must make some sort of compromise. Two
of the most common approaches are given below
The differences between these level selection
methods is illustrated by the accompanying
figure.
41
Anisotropy MIP-Mapping

• What happens when the surface is tilted?

MIP Mapped (Bi-Linear)
Nearest Neighbor
42
Questions?
43
Elliptical weighted average

• Isotropic filter wrt screen space
• Becomes anisotropic in texture space
• e.g. use anisotropic Gaussian
• Called Elliptical Weighted Average (EWA)

44
EWA resampling
45
Image Quality Comparison

• Trilinear mipmapping

EWA
trilinear mipmapping
46
Approximation of anisotropic

• Feline Fast Elliptical Lines for Anisotropic
  Texture Mapping Joel McCormack, Ronald Perry,
  Keith I. Farkas, and Norman P. Jouppi SIGGRAPH
  1999
• Andreas Schilling, Gunter Knittel Wolfgang
  Strasser. Texram A Smart Memory for Texturing.
  IEEE Computer Graphics and Applications, 16(3)
  32-41, May 1996.
• Aproximate Anisotropic Gaussian by a set of
  isotropic probes

47
FELINE results
48
Questions?
49
What's Missing?

• What's the difference between a real brick wall
  and a photograph of the wall texture-mapped onto
  a plane?
• What happens if we change the lighting or the
  camera position?

50
Bump Mapping

• Use textures to alter the surface normal
• Does not change the actual shape of the surface
• Just shaded as if it were a different shape

Swirly Bump Map
Sphere w/Diffuse Texture Bump Map
Sphere w/Diffuse Texture
51
Bump Mapping

• Treat the texture as a single-valued height
  function
• Compute the normal from the partial derivatives
  in the texture

52
Another Bump Map Example
Bump Map
Cylinder w/Diffuse Texture Map
Cylinder w/Texture Map Bump Map
53
Normal mapping

• Encodes full normal, not height
• In local coordinates (works with skinning)

actual geometry
with normal mapping
normal map
http//www.highend3d.com/maya/tutorials/texturing/
229-2.html
54
Normal map vs. bump map

• Bump map
• texture (greyscale) encodes height
• Modifies the geometric normal
• Harder to implement
• Easier to specify
• Normal map
• texture (RGB) encodes normal directly
• Replaces the normal
• but local coordinates
• Easier to implement
• Harder to specify

55
Questions?
56
What's Missing?

• There are no bumps on the silhouette of a
  bump-mapped object
• Bump maps dont allow self-occlusion or
  self-shadowing

57
Displacement Mapping

• Use the texture map to actually move the surface
  point
• The geometry must be displaced before visibility
  is determined
• Must be done as preprocess, not in hardware

58
Displacement Mapping
Image from Geometry Caching for Ray-Tracing
Displacement Maps by Matt Pharr and Pat
Hanrahan. note the detailed shadows cast by
the stones

59
Question?
60
Texture Maps for Illumination

• Also called "Light Maps"
• often different resolution than other textures

Quake
61
Question?
62
Environment Maps

• We can simulate reflections by using the
  direction of the reflected ray to index a
  spherical texture map at "infinity".
• Assumes that all reflected rays begin from the
  same point.

63
What's the Best Chart?
Slide Jan Kautz
64
Environment Mapping Example

• Terminator II

65
Questions?
66
Prefiltered environment mapping
67
Prefiltered environment mapping
Slide Jan Kautz
68
With Fresnel
69
Extreme Diffuse
70
next step replace Spherical Harmonics

• Diffuse prefiltered map is very very smooth
• Can be represented by small set of basis
  functions
• Spherical harmonics Fourier equivalent on sphere

Spherical harmonic bases Plots by Sylvain Paris
71
Question?
72
Careful antialiasing

• texture antialiasing is (semi) simple with
  diffuse shading
• Much much harder for bump mapping, environment
  mapping, etc.
• Because there is a non-linearity

73
Question?