CS 395: Adv. Computer Graphics

1
CS 395 Adv. Computer Graphics

• Introduction to
• Image-Space Methods
• Watt Watt Chapter 14 readings
• Jack Tumblin
• jet_at_cs.northwestern.edu

2
Image Space Methods

• A Digital Image a data structure for
  displayed images
• ?What can it do, besides display?
• what can we change?
• Position offset, stretch, squeeze, smear,
  cut, Try It! (Java Applets) http//www.mrl.nyu.
  edu/hertzman/nudge/
• Combine Add, Subtract/ Overlay / Compositing
• Separate matte or segmenting
• Elaborate make more of the same picture

3
Real Images Lens-Focused Light

• 3D?2D Light Intensity Map I(x,y)
• Angle(?,?) ?? Position(x,y)
• Blurringsharpness set by focus, lens quality

I(x,y) Image Plane Intensity
Viewed Scene
Angle(?,?)
Position(x,y)
4
Digital Images 2D Grid of Numbers

• NO intrinsic meaning, but ...
• Widely assumed to represent
• Point Samples of a smoothed 2D intensity
  surface
• Uniform sampling pattern (but not always)

(!weasel-word!)
y
x
5
Real vs. Digital

• Real Image 2D Light Intensity map I(x,y)
• Digital Image 2D grid of numbers I(m,n)

(pixels)
I(x,y)
I(m,n)
x,y
m,n
6
Digital Images As Vectors

• Stack up pixel values VERY LONG vector
• 1 digital image 1 point in N-dim. Space
• Nearby points Similar images
• All possible digital images a grid of N-D
  points
• All possible real images
• Unbounded dimensions
• no intensity quantizing!

I00 I01 I02 I10 I11 I12
I
I00 I01 I02 I03 I04 I05
7
Digital Images As Vectors

• Sensible element-by-element operations
• Add, subtract, scale two images
• 0.5 (I1 - I2 ) out

-

I1
I2
out
8
Digital Images As Vectors

• Sensible element-by-element operations
• ?-Weighted sum of two images 0 lt ? lt 1

? I1
(1-?) I2
out
9
Digital Images As Vectors

• Compositing
• ? Use an ? image
• as opacity

out
(1-?) . I2
? . I1
10
! NOT EQUIVALENT !

• ? Digital image pixel-by-pixel operations ?
• ? Real Image point-by-point operations ?
• Source of much frustration error!
• Digital Image Real Image
• Also see Alvy Ray Smiths article
  entitled A pixel is NOT a little square,
• a pixel is NOT a little square,
• a pixel is NOT a little square! found
  here.

11
Real Digital

• ?

• Image Intensity
• Image Spectral Distribution (color)
• Image Geometry (Position,Angle, Distortion, )
• Image Resolution (sharpness or focus)
• Real Images Smooth,
• Continuous,
• Variable,
• Complete
• Unlimited

• ?
• ?
• ?
• ?

• Digital Images
• Sampled,
• Discrete,
• Quantized,
• Fixed
• Limited

12
Conversion Method?

• Real Image 2D Light Intensity map I(x,y)
• Digital Image 2D grid of numbers I(m,n)

• ??

13
Digital ? Real Reconstruction

• A pixel sets strength of a real Basis
  function (aka reconstruction filters
  impulse response)

Box
I(m,n)
I(x,y)
m,n
x,y
14
Digital ? Real Reconstruction

• A pixel sets strength of a real Basis
  function (aka reconstruction filters
  impulse response)

Linear
1.0
I(m,n)
I(x,y)
0
-1.0
1.0
m,n
x,y
15
Digital ? Real Reconstruction

• A pixel sets strength of a real Basis
  function (aka reconstruction filters
  impulse response)

Cubic
1.0
I(m,n)
I(x,y)
0
-1.0
1.0
2.0
m,n
x,y
16
Real?Digital Sampling

• If I(x,y) is smooth enough already,
• ?Why not just grab I(x,y) values at (m,n) ?

I(m,n)

• ?

m,n
17
Smooth enough to Sample?

• Because smooth enough is undefined
• Samples may hit spurious peaks, valleys

I(x,y)
I(m,n)
BAD!
x,y
m,n
18
Real ? Digital Pre-filter

• Smoothing Linear Pre-filter
• Pixel local weighted average of real
  image around pixel sample point

Weight
1.0
Cubic weights
I(x,y)
1.0
0
-1.0
1.0
2.0
x,y
19
Pre-Filter functions

• Pre-filters anti-aliasing in Comp Graphics
• SHOULD be done by
• continuous integration,
• USUALLY done by
• super-sampling
• ? Quality Measures ?

Box filter
Bi-linear filter
Bi-cubic filter
20
Smoothness Measures

• Sine wave magic
• ALL (continuous) functions are a (possibly
  infinite) weighted sum of sinusoids f(x)
  A0sin(w0x) B0cos(w0x) A1sin(w1x)
  B1cos(w0x)
• For ALL FILTERS (box, linear, cubic,,
  anything)sinusoid in ? scaled, shifted sinusoid
  out
• High-Frequency Sinusoids hold ALL the sampling
  trouble!

21
1-D Aliasing Example
Real
Real image I(x,y) sin(2?fx) f3.0
Aliasing when bad conversion makes scrambles
the frequency of sinusoids
22
1-D Aliasing Example
Real
Sample
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per cycle)
23
Aliasing Example
Real
Sample
Display
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per
cycle) Reconstruct (use box filter) ?
approximates f3.0
24
Aliasing Example
Real
Sample
Display
Real
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per
cycle) Reconstruct (use box filter) ?
approximates f3.0 Another Real Image I(x,y)
sin(2?fx) where f 39.0
25
Aliasing Example
Real
Sample
Display
Sample
Real
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per
cycle) Reconstruct (use box filter) ?
approximates f3.0 Another Real Image I(x,y)
sin(2?fx) where f 39.0 Sample at x(n) n /
36.0
26
Aliasing Example
Real
Sample
Display
Display
Sample
Real
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per
cycle) Reconstruct (use box filter) ?
approximates f3.0 Another Real Image I(x,y)
sin(2?fx) where f 39.0 Sample at x(n) n /
36.0 Reconstruct (use box filter)?
approximates f3.0 !!
27
Aliasing Example
Real
Sample
Display
Display
Sample
Real
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per
cycle) Reconstruct (use box filter) ?
approximates f3.0 Another Real Image I(x,y)
sin(2?fx) where f 39.0 Sample at x(n) n /
36.0 Reconstruct (use box filter)?
approximates f3.0
28
Aliasing Example
Real
Sample
Display
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per
cycle) Reconstruct (use box filter) ?
approximates f3.0 Another Real Image I(x,y)
sin(2?fx) where f 39.0 Sample at x(n) n /
364 n / 144.0 Reconstruct (use box filter)?
approximates f39.0
29
Conclusion

• Image-space manipulations must explicitly address
  real-digital (discrete-continuous) conversions
  to avoid aliasing artifacts.
• Digital images are compact, indirect, ambiguous
  descriptions of light distributions

30
Image Space Methods

• Outline
• Definitions, limitations (Whats a Digital
  Image? Can they really represent ANY image?
  (no).
• The Image Vector
• What can we do with Digital Images?
• Intensity math, editing
• Image Comparisons / Similarity Measures
• Pattern finding, matching, learning,
• Spatial math, editing
• Pitfall Real ??Digital Images
  Discrete ?? Continuous Representations
• Conversions
• Reconstruction digital?real each pixel
  scales a basis function
• Sampling real?digital each found by basis
  function (wieghted sum of real image
  neighborhood)

31
Image Space Methods

• Outline (contd)
• Aliasing Artifacts Bad Conversions
  Somewhere not enough smoothing
• Sine Wave Magic!
• Fouriers Big Idea 1 all functions are a unique
  weighted sum of sinusoids! (square wave example)
• Fouriers Big Idea 2 no weighted-average filter
  can change a sinusoids shape! (sine wave in ?
  sine-wave out (but it might be shifted scaled))
• Infinite sharpness? Infinitely high-frequ
• Why does it matter? We cant reconstruct a
  sine-wave changing gt ½ sample rate.
• Math jargon sinewaves are eigenfunctions of the
  space of all linear functionals.
• Suppose you plot the amount of shift amount of
  scale for all possible sinusoids Fourier
  Transform!

32
Aliasing Example
Real
Sample
Display
Real
Sample
Display
Real image I(x,y) sin(2?fx) f3.0 Sample at
x(n) n / 36.0 (12 samples per
cycle) Reconstruct (use box filter) ?
approximates f3.0 Another Real Image I(x,y)
sin(2?fx) where f 39.0 Sample at x(n) n /
36.0 Reconstruct (use box filter)?
approximates f3.0
33
SAFETY COPY
34
Reconstruction Digital ? Real

• How to Read Between the Pixels

35
Pre-Filter Real ? Digital
36
Image Space Methods

• Intro What can we do with images?
• Re-samplingContinuous Discrete Filters
• Interpolation Whats in-between a pixel?
• Antialiasing What makes a good sampling?
• Math Rigor the Fourier Transform
• Examples box, triangle, Mitchell-Netravali
• Compositing Transparency
• Un-Compositing Matte Separations
• Further extensions
• Environment Matte
• Polynomial Texture Map
• Texture Elaboration, Image Inpainting(Sigg2000)