FRS 123: Technology in Art and Cultural Heritage

1
FRS 123 Technology inArt and Cultural Heritage

• Perception

2
Low-Level or Early Vision

• Considers local properties of an image

Theres an edge!
3
Mid-Level Vision

• Grouping and segmentation

Theres an object and a background!
4
High-Level Vision

• Recognition

Its a chair!
5
Image Formation

• Human lens forms image on retina,sensors (rods
  and cones) respond to light
• Computer lens system forms image,sensors (CCD,
  CMOS) respond to light

6
Intensity

• Perception of intensity is nonlinear

7
Modeling Nonlinear Intensity Response

• Perceived brightness (B) usually modeled as a
  logarithm or power law of intensity (I)
• Exact curve varies with ambient light,adaptation
  of eye

8
CRT Response

• Power law for Intensity (I) vs.applied voltage
  (V)
• Other displays (e.g. LCDs) contain electronics to
  emulate this law

9
Cameras

• Original cameras based on Vidicon obey power law
  for Voltage (V) vs. Intensity (I)
• Vidicon CRT almost linear!

10
CCD Cameras

• Camera gamma codified in NTSC standard
• CCDs have linear response to incident light
• Electronics to apply required power law
• So, pictures from most cameras (including digital
  still cameras) will have g 0.45

11
Contrast Sensitivity

• Contrast sensitivity for humans about 1
• 8-bit image (barely) adequate if using perceptual
  (nonlinear) mapping
• Frequency dependent contrast sensitivity lower
  for high and very low frequencies

12
Contrast Sensitivity

• Campbell-Robson contrast sensitivity chart

13
Bits per Pixel Scanned Pictures
8 bits / pixel / color
6 bits / pixel / color
Marc Levoy / Hanna-Barbera
14
Bits per Pixel Scanned Pictures (cont.)
5 bits / pixel / color
4 bits / pixel / color
Marc Levoy / Hanna-Barbera
15
Bits per Pixel Line Drawings
8 bits / pixel / color
4 bits / pixel / color
Marc Levoy / Hanna-Barbera
16
Bits per Pixel Line Drawings (cont.)
3 bits / pixel / color
2 bits / pixel / color
Marc Levoy / Hanna-Barbera
17
Color

• Two types of receptors rods and cones

Rods and cones
Cones in fovea
18
Rods and Cones

• Rods
• More sensitive in low light scotopic vision
• More dense near periphery
• Cones
• Only function with higher light
  levelsphotopic vision
• Densely packed at center of eye fovea
• Different types of cones ? color vision

19
Electromagnetic Spectrum

• Visible light frequencies range between ...
• Red 4.3 x 1014 hertz (700nm)
• Violet 7.5 x 1014 hertz (400nm)

20
Color Perception

• 3 types of cones L, M, S

M
L
Tristimulus theory of color
S
21
Tristimulus Color

• Any distribution of light can be summarized by
  its effect on 3 types of cones
• Therefore, human perception of color is
  a3-dimensional space
• Metamerism different spectra, same response
• Color blindness fewer than 3 types of cones
• Most commonly L cone M cone

22
Color CRT
23
Preattentive Processing

• Some properties are processed preattentively
• (without need for focusing attention).
• Important for art, design of visualizations
• what can be perceived immediately
• what properties are good discriminators
• what can mislead viewers

Preattentive processing sildes from
Healeyhttp//www.csc.ncsu.edu/faculty/healey/PP/P
P.html
24
Example Color Selection
Viewer can rapidly and accurately
determine whether the target (red circle) is
present or absent. Difference detected in color.
25
Example Shape Selection
Viewer can rapidly and accurately
determine whether the target (red circle) is
present or absent. Difference detected in form
(curvature)
26
Pre-attentive Processing

• lt 200250 ms qualifies as pre-attentive
• eye movements take at least 200ms
• yet certain processing can be done very quickly,
  implying low-level processing in parallel
• If a decision takes a fixed amount of time
  regardless of the number of distractors, it is
  considered to be preattentive

27
Example Conjunction of Features
Viewer cannot rapidly and accurately
determine whether the target (red circle) is
present or absent when target has two or more
features, each of which are present in the
distractors. Viewer must search sequentially.
28
Example Emergent Features
Target has a unique feature with respect to
distractors (open sides) and so the group can be
detected preattentively.
29
Example Emergent Features
Target does not have a unique feature with
respect to distractors and so the group cannot
be detected preattentively.
30
Asymmetric and Graded Preattentive Properties

• Some properties are asymmetric
• a sloped line among vertical lines is
  preattentive
• a vertical line among sloped ones is not
• Some properties have a gradation
• some more easily discriminated among than others

31
SUBJECT PUNCHED QUICKLY OXIDIZED TCEJBUS DEHCNUP
YLKCIUQ DEZIDIXO CERTAIN QUICKLY PUNCHED METHODS
NIATREC YLKCIUQ DEHCNUP SDOHTEM SCIENCE ENGLISH
RECORDS COLUMNS ECNEICS HSILGNE SDROCER
SNMULOC GOVERNS PRECISE EXAMPLE MERCURY SNREVOG
ESICERP ELPMAXE YRUCREM CERTAIN QUICKLY PUNCHED
METHODS NIATREC YLKCIUQ DEHCNUP SDOHTEM GOVERNS
PRECISE EXAMPLE MERCURY SNREVOG ESICERP ELPMAXE
YRUCREM SCIENCE ENGLISH RECORDS COLUMNS ECNEICS
HSILGNE SDROCER SNMULOC SUBJECT PUNCHED QUICKLY
OXIDIZED TCEJBUS DEHCNUP YLKCIUQ
DEZIDIXO CERTAIN QUICKLY PUNCHED METHODS NIATREC
YLKCIUQ DEHCNUP SDOHTEM SCIENCE ENGLISH RECORDS
COLUMNS ECNEICS HSILGNE SDROCER SNMULOC
32
Text NOT Preattentive
SUBJECT PUNCHED QUICKLY OXIDIZED TCEJBUS DEHCNUP
YLKCIUQ DEZIDIXO CERTAIN QUICKLY PUNCHED METHODS
NIATREC YLKCIUQ DEHCNUP SDOHTEM SCIENCE ENGLISH
RECORDS COLUMNS ECNEICS HSILGNE SDROCER
SNMULOC GOVERNS PRECISE EXAMPLE MERCURY SNREVOG
ESICERP ELPMAXE YRUCREM CERTAIN QUICKLY PUNCHED
METHODS NIATREC YLKCIUQ DEHCNUP SDOHTEM GOVERNS
PRECISE EXAMPLE MERCURY SNREVOG ESICERP ELPMAXE
YRUCREM SCIENCE ENGLISH RECORDS COLUMNS ECNEICS
HSILGNE SDROCER SNMULOC SUBJECT PUNCHED QUICKLY
OXIDIZED TCEJBUS DEHCNUP YLKCIUQ
DEZIDIXO CERTAIN QUICKLY PUNCHED METHODS NIATREC
YLKCIUQ DEHCNUP SDOHTEM SCIENCE ENGLISH RECORDS
COLUMNS ECNEICS HSILGNE SDROCER SNMULOC
33
Preattentive Visual Properties Healey 97

• length Triesman
  Gormican 1988
• width Julesz
  1985
• size
  Triesman Gelade 1980
• curvature Triesman
  Gormican 1988
• number Julesz
  1985 Trick Pylyshyn 1994
• terminators Julesz
  Bergen 1983
• intersection Julesz
  Bergen 1983
• closure Enns
  1986 Triesman Souther 1985
• colour (hue) Nagy
  Sanchez 1990, 1992 D'Zmura 1991
  Kawai
  et al. 1995 Bauer et al. 1996
• intensity Beck et
  al. 1983 Triesman Gormican 1988
• flicker Julesz
  1971
• direction of motion Nakayama
  Silverman 1986 Driver McLeod 1992
• binocular lustre Wolfe
  Franzel 1988
• stereoscopic depth Nakayama
  Silverman 1986
• 3-D depth cues Enns 1990
• lighting direction Enns 1990

34
Accuracy Ranking of Quantitative Perceptual
TasksEstimated only pairwise comparisons have
been validatedMackinlay 88 from Cleveland
McGill
35
Visual Illusions

• People dont perceive length, area, angle,
  brightness they way they should
• Some illusions have been reclassified
  assystematic perceptual errors
• e.g., brightness contrasts (grey square onwhite
  background vs. on black background)
• partly due to increase in our understanding
  ofthe relevant parts of the visual system
• Nevertheless, the visual system does some really
  unexpected things

36
Illusions of Linear Extent

• Mueller-Lyon (off by 25-30)
• Horizontal-Vertical

37
Illusions of Area

• Delboeuf Illusion
• Height of 4-story building overestimated by
  approximately 25

38
Low-Level Vision
Hubel
39
Low-Level Vision

• Retinal ganglion cells
• Lateral Geniculate Nucleus visual adaptation?
• Primary Visual Cortex
• Simple cells orientational sensitivity
• Complex cells directional sensitivity
• Further processing
• Temporal cortex what is the object?
• Parietal cortex where is the object? How do I
  get it?

40
Low-Level Vision

• Net effect low-level human visioncan be
  (partially) modeled as a set ofmultiresolution,
  oriented filters

41
Low-Level Computer Vision

• Filters and filter banks
• Implemented via convolution
• Detection of edges, corners, and other local
  features
• Can include multiple orientations
• Can include multiple scales filter pyramids
• Applications
• First stage of segmentation
• Texture recognition / classification
• Texture synthesis

42
Texture Analysis / Synthesis
Multiresolution Oriented Filter Bank
OriginalImage
Image Pyramid
43
Texture Analysis / Synthesis
Original Texture
Synthesized Texture
Heeger and Bergen
44
Low-Level Computer Vision

• Optical flow
• Detecting frame-to-frame motion
• Local operator looking for gradients
• Applications
• First stage of tracking

45
Optical Flow
Image 1
Optical FlowField
Image 2
46
Low-Level Depth Cues

• Focus
• Vergence
• Stereo
• Not as important as popularly believed

47
3D Perception Stereo

• Experiments show that absolute depth estimation
  not very accurate
• Low relief judged to be deeper than it is
• Relative depth estimation very accurate
• Can judge which object is closer for stereo
  disparities of a few seconds of arc

48
3D Perception Illusions
Block Yuker
49
3D Perception Illusions
Block Yuker
50
3D Perception Illusions
Block Yuker
51
3D Perception Illusions
Block Yuker
52
3D Perception Illusions
Block Yuker
53
3D Perception Illusions
Block Yuker
54
3D Perception Illusions
Block Yuker
55
3D Perception Illusions
Block Yuker
56
3D Perception Illusions
Block Yuker
57
3D Perception Illusions
Block Yuker
58
3D Perception Conclusions

• Perspective is assumed
• Relative depth ordering
• Occlusion is important
• Local consistency

59
Low-Level Computer Vision

• Shape from X
• Stereo
• Motion
• Shading
• Texture foreshortening

60
3D Reconstruction
TomasiKanade
Forsyth et al.
Phigin et al.
Debevec,Taylor,Malik
61
Mid-Level Vision

• Physiology unclear
• Observations by Gestalt psychologists
• Proximity
• Similarity
• Common fate
• Common region
• Parallelism
• Closure
• Symmetry
• Continuity
• Familiar configuration

Wertheimer
62
Gestalt Properties

• Gestalt form or configuration
• Idea forms or patterns transcend thestimuli
  used to create them
• Why do patterns emerge? Under what circumstances?

Why perceive pairs vs. triplets?
63
Gestalt Laws of Perceptual Organization Kaufman
74

• Figure and Ground
• Escher illustrations are good examples
• Vase/Face contrast
• Subjective Contour

64
More Gestalt Laws

• Law of Proximity
• Stimulus elements that are close together will be
  perceived as a group
• Law of Similarity
• like the preattentive processing examples
• Law of Common Fate
• like preattentive motion property
• move a subset of objects among similar ones and
  they will be perceived as a group

65
Grouping Cues
66
Grouping Cues
67
Grouping Cues
68
Grouping Cues
69
Events of Interest

• /_at_rts lecture series on interrelations of new
  media, technology and traditional forms and
  practices of arts and humanities
  http//www.princeton.edu/slasharts/
• Scott McCloud Comics An Art Form in
  Transition Thursday, October 5, 430
  pm Jimmy Stewart Theater 185 Nassau Street

70
Events of Interest

• Digital Stone Project
• Local facility for automated creation of stone
  sculpture from 3D computer models
• Computer-contolled milling machines, lathes
• Friday, October 6, 130-400Meet in Computer
  Science building,2nd floor tea room, 100 sharp