Paint By Numbers

1
Paint By Numbers

• The goal of a visual artist (Hagen)
• Without modeling detail, painters use brush
  strokes to
• Represent objects
• Direct attention

The goal of effective representational image
making, whether you paint in oil or in numbers,
is to select and manipulate visual information in
order to direct the viewers attention and
determine the viewers perception.
2
Basic Approach

• Aim To take a synthetic or natural scene and
  convert it into an abstract representation
• The paper describes an interactive tool
• A user drags the mouse over a source image
• The image is point sampled, and the color drives
  a brush stroke
• Avoids explicit color selection common in paint
  programs
• Controlling the nature of the strokes controls
  the form of the picture

3
Representation

• A painting is an ordered list of brush strokes
• A brush stroke is described by
• Location
• Color
• Size
• Direction
• Shape
• The aim is to use the strokes to convey
• Surface color
• Surface curvature
• Focus
• Edges

4
Controlling the Brush Strokes

• Position comes from mouse with random noise added
• Color comes from source image, with random noise
  added
• Size comes from the speed of mouse motion or
  explicit user selection
• Stroke direction comes from mouse motion
  direction or explicit user selection
• Shape is chosen by the user (circles, rectangles,
  lines, scatterings, user design)

5
Post Processing Operations

• All brush strokes can be stored and later
  manipulated
• Operations include
• Add noise to some component
• Simple animation by applying same stroke
  parameters to sequence of images, choosing color
  from each image
• Use gradient to set orientation
• Use second derivative to enhance edges

6
Some Other Things

• Use ray tracing to extract color from a 3d model
• Apply simple stochastic gradient decent to
  automatically place strokes
• Fix the number of strokes
• Repeatedly Perturb an attribute, of the result
  is better keep the change
• Using cone shaped brushes produces Voronoi
  diagrams (cute observation, completely unrelated
  to NPR)

7
Computer-Generated Pen-and-Ink Illustration

• Illustrations offer many benefits over
  traditional rendering
• Detail can be enhanced, or omitted
• Vitality and style
• Applications
• Medical illustration, Workshop manuals, Design
  sketches
• Pen-and-Ink Features
• Easy and cheap to print, and goes well with text
• Color and shading must be implied by stroke
  combinations

8
Strokes

• A curve segment traced out by a pen
• Principles
• Too thin is washed out too thick obscures detail
• The pen may be turned or varied during the stroke
• Strokes should look natural - to add life.
  Characteristics, such as width, should be varied
• Wavy lines indicate immature ideas (sketches)

9
Tones and Texture

• Combinations of strokes combine to form
• Tone the brightness of a point
• Texture an indication of the surface properties
• Principles
• Roughly equal weight and spacing for tones
• Absolute values of tones is not really important,
  but differences in tone are very important
• Tones should sometimes be forced to disambiguate
  objects

10
More Tones and Textures

• Texture principles
• The character of strokes is important for texture
• Crisp straight lines for glass
• Horizontal surfaces with horizontal lines
• Absence of detail indicates glare
• Sketchy indicates old, crisp indicates new
• To lend economy to the image, texture should be
  indicated
• Only draw some subset of strokes
• Vary the exact method across the illustration

11
Outlines

• Outlines are completely abstract - photographs
  dont have them
• Not just for contours - may also indicate
  interior detail (veins in a leaf)
• Principles
• Outline supports textures
• Thick lines for shadows, or less depth
• Haloes where objects pass behind each other
• Outlines where tone is omitted to convey shape
• Indication is just as important as it is for
  texture

12
Input and Preprocessing

• A 3D model
• Associate texture with each face
• Render with diffusespecular to get reference
  tones
• Indicate where detail should appear
• Indicate where the light is
• Generate shadow polygons using a BSP tree shadow
  volume algorithm
• Generate a map of the screen, indicating what
  appears where, and the relative depths

13
Laying Down Tone and Texture

• Fill each region with stroke texture
• Set of strokes with priorities as to which should
  be rendered first
• Scale independent - appropriate strokes get drawn
  depending on resolution and size
• User indicated detail area drive placement
• Field describes how much texture should be drawn
• Field drops off away from detail edges
• Strokes are clipped to regions, with some noise

14
Laying Down Outlines

• Outline associated with each texture
• If shading indicates an edge, do not drawn
  outline
• Where surfaces meet, use closest texture as
  outline
• Thick or thin lines to indicate shadow and
  relation to the light sources
• Dependence on viewing direction
• View direction affects which strokes are drawn

15
Results
16
Still There to Solve

• Automatic emphasis control
• Rendering other types of scenery (such as trees,
  to be presented later)
• Animating pen-and-ink
• Coherence
• Adding and dropping strokes as priorities change

17
Automatic Technical Illustration
The underlying assumption in NPR is that artistic
techniques developed by human artists have
intrinsic merit based on the evolutionary nature
of art.

• Concentrates on shading, assuming edge lines are
  desirable and will be added
• Key techniques
• Cool-to-warm shading
• Metal shading

18
Traditional Shading

• Assume highlights and edge lines are also added
• In shadow areas, ambient term dominates
• Ambient must be high to contrast edge lines
• No detail in shadow areas
• In areas facing light source, brightest diffuse
  color dominates
• No contrast with highlights
• Have to reduce kd
• Detail is lost!

19
Tone-based shading

• Tones come from adding gray to a color
• Shifts hue, but leaves luminance roughly intact
• Temperature of a color is
• High if color is close to red
• Low if color is close to blue
• Hot colors appear to be forward, while cold
  colors appear to be back
• Algorithm
• Replace cosine term in regular model with a new
  term that blends between hot and cold

20
New Model

• Regardless of base color, resulting tones will be
  cold or warm
• Then use
• Variation even in shadow regions
• Robust to parameter settings

21
Metal Shading

• Traditional methods indicate metal by alternating
  dark and light bands
• Bands are aligned with direction of minimum
  curvature (so along the axis of a cylinder)
• Derived from anisotropic reflection of milled
  objects
• Modeled by adding 20 stripes around the object,
  and a bright stripe closest to the light source

22
Real-Time Approximations

• Aim To approximate the cool-to-warm shading
  model with standard rendering hardware
• Use two lights to get cool-to-warm
• Warm light from one direction, cool light from
  opposite direction
• Exploits negative intensity light sources
• Stencil buffers or software to get contours and
  silhouettes
• Texture or environment maps to get metal shading
• Rotate the object, keeping the viewer and light
  fixed

23
More Open Problems

• They do not exploit cool-to-warm to help with
  depth perception. Can that be done?
• Related to another problem Contrast
  maintenance/enhancement in tone reproduction
  algorithms
• Basic idea Adjust the shading to enhance local
  features, while maintaining some global
  consistency