Object recognition using shading

1
Object recognition using shading

2
Object recognition using shading

• Strong claim 1 to recognize an object using
  shading, we reconstruct the objects 3D shape
  and then we recognize this 3D shape.

3
Object recognition using shading

• Strong claim 2 shape from shading is not
  used in object recognition

4
What is the role of shape from shading in object
recognition ?

• To what extent do humans perceive 3D shape from
  shading ?
• To what extent do humans use 3D shape perception
  to recognize objects ?
• We need to answer Q1 before we can answer Q2.

5
Human perception of local shape from shading
under variable lighting

• Michael S. Langer Heinrich H. Bülthoff
• Max-Planck-Institute for Biological
  CyberneticsTübingen, Germany
• McGill University, Montreal, Canada

6
Illumination models

• Sunny day (Horn 70, )
• Cloudy day (Langer and Zucker ICCV 93,
  Stewart and Langer CVPR 97)

7
Overview

• Experiment 1 SFS on a sunny day
• Experiment 2 SFS on a cloudy day

8
SFS on a Sunny Day
L
N(x)
I(x) r N(x) L
9
Depth-reversal ambiguity on a sunny day
hill
valley
10

• Hollow Mask Illusion (Luckiesh, 1916)

11
Hollow mask illusion is due to two
factors(Johnston et al 92 , Hill and Bruce
94)

• face vs. hollow mask
• familiarity global
  convexity
• (recognition)

12
Our experiments use unfamiliar surfaces

• convex
  concave
• face
  mask

13
Procedure

14

15
Task hill or valley ?

16
Three factors were tested

• light source direction (above gt below)
• global shape (convex gt
  concave)
• viewpoint (above gt
  below)

17
1. light source direction

• light
• from
• above
• light
• from
• below
• 
  

18
2. global shape

• convex
  concave

19
3. Viewpoint (Reichel Todd 1990)

• from above from
  below
• (floor)
  (ceiling)

20
Factor 3 viewpoint

• view from above
• 
  
  
• view from below
• 
  

21
Factor 3 viewpoint

• view from below
• 
  
  
• view from above
• 
  

22
Design

• three factors (2 x 2 x 2)
• - light direction (above, below)
• - global shape (convex, concave)
• - viewpoint (floor, ceiling)
• 512 trials (64 x 8 conditions)

23
Results (linear regression)

• percent correct (hill or valley ?)
• 51 10 light direction
• 13 global shape
• 11 viewpoint
• (Each factor had value of 1 or
  1)

24
Conclusions Experiment 1

• many factors play a role in resolving the
  depth-reversal ambiguity
• (light direction, global shape, viewpoint,
  .)

25
Conclusions Experiment 1

• many factors play a role in resolving the
  depth-reversal ambiguity
• (light direction, global shape, viewpoint,
  .)
• observers often ignore available image
  information (perspective, occluding contours,
  shadows)

26
Experiment 2 Shape from shading on a cloudy day
27
Shading on a cloudy day
?
(x)
28
Shading on a Cloudy Day
?
?
I(x)
N(x)
L d L
?
(x)
?
?(x) angle of visible light source
29
Shading in a valley on a cloudy day
local intensity maxima
30
Local intensity maxima in valleys
31
Experiment 2
How well do humans perceive shape from shading in
the presence of these local intensity maxima ?
32
Experiment 2
How well do humans perceive shape from shading in
the presence of these local intensity maxima
? Hypothesis (shape from shading
skeptic) Humans use dark means deep model.
33
Procedure
34
(No Transcript)
35
Task Which is higher ?
36
Two conditions
???intensity
correlated
anti-correlated
_

???height
_

anti-correlated
correlated
37
Which is higher ?
N17
overall score 65 (above chance)
percent correct
_

38
Conclusion Experiment 2

• Dark means deep is too simple a model to
  explain human perception of shape from shading
  on a cloudy day.

39
Big Picture What role does 3D shape perception
play in 3D object recognition?
40
Big Picture What role does 3D shape perception
play in 3D object recognition?

• To what extent do we perceive 3D shape ? Were
  on the way to answering this question.

41
Big Picture What role does 3D shape perception
play in 3D object recognition?

• To what extent do we perceive 3D shape ? Were
  on the way to answering this question.
• To what extent do we use these 3D shape percepts
  to recognize objects ?
• The answer to Q2 depends on Q1.

42
Computer vision psychophysics
I(x) q(x)

(Langer and Zucker 93)
?
I(x) N(x) L d L
?
?(x)
(Stewart and Langer 96)
43
PsychophysicsHuman vs. Computer
1
0.8
q q, N
0.6
percent correct
0.4
0.2
-
0

-

-

human computer
44
Conclusions Experiment 2

• Dark means deep is too simple to explain human
  perception of shape from shading on a cloudy day
• Computing local shape in valleys is an inherently
  difficult computational problem on a cloudy
  day

45
Which point is brighter ?
height
N10
brightness
percent correct
_

Correlation
46
Examples

87 (best)


15 (worst)