The Pulfrich effect: A new explanation of an old illusion

1
The Pulfrich effectA new explanation of an old
illusion

• Jenny Read

Psychology Division Newcastle University
2
Outline

• Brief background on stereopsis
• What is the Pulfrich effect?
• Previous explanations of Pulfrich effect
• Our new explanation
• How psychophysics, physiology modelling combine
  to explain the neural basis of this illusion.

Acknowledgements Bruce Cumming, Laboratory of
Sensorimotor Research, National Institutes of
Health, Bethesda, Maryland.
3
Put red lens over left eye, blue lens over right
eye Stereo anaglyph by Prof. Michael
Greenhalgh, Australian National University (with
permission).
4
stereopsis
mountain
tree
5
stereopsis
mountain
tree
6
stereopsis
mountain
tree
7
stereopsis
mountain
tree
8
stereopsis
mountain
tree
9
how the glasses work
reproduces the different views each eye would
have seen if the objects had really been at
different distances.
10
how the glasses work
11
how the glasses work
screen
12
how the glasses work
screen
screen
13
how the glasses work
screen
14
how the glasses work
screen
15
how the glasses work
screen
16
how the glasses work
17
demo of Pulfrich illusion

• Pendulum going back and forth on screen
• Viewed with one eye dimmed.
• This introduces interocular delay.
• You see it rotate in depth

18
The Pulfrich effect
Reality Perception

• Illusory perception of a moving object when one
  eyes image is delayed

19
Space-time diagram
space
time

• Moving object

20
The Pulfrich effect
left
space
right
time

• Moving object.
• A delay is introduced in one eyes image.

21
The Pulfrich effect
left
space
right
time

• Moving object.
• A delay is introduced in one eyes image.
• The object is perceived as moving in depth.

22
The original explanation

• Spatial disparity and temporal delay are
  geometrically equivalent.

23
The Pulfrich effect
left
space
right
time

• Spatial disparity and temporal delay are
  geometrically equivalent.

24
spatial disparity and temporal delay are
equivalent
fovea
25
spatial disparity and temporal delay are
equivalent
left eye
F
F
right eye
26
zero spatial disparity
Here the left and right images fall at the same
distance from the fovea in both retinae.
left eye
F
F
right eye
27
moving object with zero spatial disparity
28
moving object with zero spatial disparity
left eye
F
F
right eye
29
moving object with zero spatial disparity
left eye
F
reitnal position
time
F
right eye
30
moving object with zero spatial disparity
left eye
F
reitnal position
time
F
right eye
31
near disparity
Now the image in the right eye is always one
slot below the image in the left eye.
left eye
F
F
right eye
32
moving object with near disparity
Now the image in the right eye is always one
slot below the image in the left eye.
left eye
F
F
right eye
33
moving object with near disparity
Now the image in the right eye is always one
slot below the image in the left eye.
left eye
F
position
time
F
right eye
34
Spatial disparity
position
time
35
Spatial disparity
Temporal delay
position
position
time
time
36
Spatial disparity
Temporal delay
position
position
time
time
37
moving object with zero spatial disparity but
with temporal delay
38
moving object with zero spatial disparity but
with temporal delay
1
2
2
1
Image 1 from the right eye reaches the brain at
the same time as image 2 from the left.
39
moving object with zero spatial disparity but
with temporal delay
1
2
2
1
The brain doesnt know about the time delay
so it deduces the object is closer than it
really is.
40
moving object with zero spatial disparity but
with temporal delay
1
3
2
3
2
41
moving object with zero spatial disparity but
with temporal delay
1
3
2
3
2
42
moving object with zero spatial disparity but
with temporal delay
1
4
2
3
4
3
43
moving object with zero spatial disparity but
with temporal delay
1
4
2
3
4
3
44
An object moving in the plane of fixation but
with interocular delay
45
is perceived as moving in a plane closer to the
observer.
46
For motion in the opposite direction, the object
is seen as further away.
47
For motion in the opposite direction, the object
is seen as further away.
48
For motion in the opposite direction, the object
is seen as further away.
49
Spatial disparity is geometrically equivalent to
interocular delay.
position
position
time
time
50
Spatial disparity is geometrically equivalent to
interocular delay.

• This explanation was accepted for nearly 100
  years
• until someone came up with the STROBOSCOPIC
  PULFRICH EFFECT.

51
demo of strobe Pulfrich
52
Stroboscopic Pulfrich effect
Flashing stimulus, one eye lagging the other.
space
time
now
53
Stroboscopic Pulfrich effect
Flashing stimulus, one eye lagging the other.
No spatial disparity, purely temporal delay.
space
no spatial disparity
time
interocular delay
54
Stroboscopic Pulfrich effect

• How to explain the perception of depth with a
  stroboscopic stimulus?
• So people suggested that the visual system may
  have sensors which detect both motion and
  disparity.

55
Left and right eye inputs are first combined in V1
56
Receptive fields

• Remember that neurons only respond to stimuli
  within their receptive field.
• We have seen how the stimulus can be represented
  in a space-time diagram

position
time
57
Receptive field
space
time
58
Receptive field
space
time
neuronal spiking
time
59
This receptive field responds equally well to
motion in either direction.
space
time
neuronal spiking
time
60
For a direction-sensitive cell, we need a tilted
receptive field.
space
time
neuronal spiking
time
61
For a direction-sensitive cell, we need a tilted
receptive field.
space
time
neuronal spiking
time
62
For a direction-sensitive cell, we need a tilted
receptive field.
space
time
neuronal spiking
time
63
Joint motion-disparity detectors

• For binocular neurons, we need a receptive field
  in each eye.
• The difference in position of the receptive field
  in each eye defines the stereo disparity to which
  the cell responds.

64
F
F
65
receptive fields tuned to near disparity
left-eye receptive field on the retina
right-eye receptive field
66
receptive fields tuned to near disparity
left-eye receptive field on the retina
right-eye receptive field
67
receptive fields tuned to near disparity
left-eye receptive field on the retina
right-eye receptive field
68
receptive fields tuned to near disparity
left-eye receptive field on the retina
right-eye receptive field
69
So, a cell which is sensitive to both motion and
disparity would have receptive fields like this
space
time
left-eye receptive field
70
These sensors would normally respond to a moving
object with disparity
space
time
now
71
They will also respond to a moving object with no
disparity but an interocular delay.
space
time
now
72
Stroboscopic Pulfrich effect

• No spatial disparity, purely interocular delay.

Stroboscopic stimulus activates tilted RFs.
space
time
now
73
Consensus ca. 2005

• Key difference between classic Pulfrich effect
  and strobe Pulfrich
• Classic Pulfrich reflects stimulus geometry
• temporal delay ? spatial disparity
• Strobe Pulfrich reflects brain mechanisms
• neurons that encode both motion and disparity
• neuronal basis of Pulfrich effect

74
electrophysiology

• Single-unit recordings in awake monkey.
• We looked for joint motion-disparity sensors in
  monkey primary visual cortex.
• We found very few!
• Most cells in V1 encode only disparity. Fewer
  encode motion, and fewer still encode motion and
  disparity together.

Read Cumming 2005 Journal of Neurophysiology
94 1541
75
a few cells do have tilted RFs
35
42
30
28
25
14
interocular delay (ms)
20
firing rate (spikes/s)
0
15
-14
10
-28
5
-42
-0.2
-0.1
0
0.1
0.2
interocular disparity (deg)
76
but most are not tilted.
77
tilted receptive fields
straight receptive fields
space
space
time
time
predicted by joint-encoding models
implied by our physiology experiments
78
electrophysiology

• So what does this mean?
• Is your perception of depth in the stroboscopic
  Pulfrich stimulus due to a tiny subset of cells
  in primary visual cortex?
• Or, maybe we dont need joint encoding after all.

79
Stroboscopic Pulfrich effect
Flashing stimulus, one eye lagging the other.
No spatial disparity, purely temporal delay.
space
time
interocular delay
80
Stroboscopic Pulfrich effect
But there are other possible matches
near disparity
space
time
interocular delay
81
Stroboscopic Pulfrich effect
But there are other possible matches
space
far disparity
time
interocular delay
82
Stroboscopic Pulfrich effect
Perhaps perception can be influenced by several
matches at once?
space
time
83
Stroboscopic Pulfrich effect
Postulate that most weight is given to matches
close together in time.
space
far disparity least weight
near disparity some weight
zero disparity most weight
time
84
Stroboscopic Pulfrich effect
Postulate that most weight is given to matches
close together in time.
space
far disparity least weight
near disparity some weight
zero disparity most weight
time
85
Stroboscopic Pulfrich effect
This could bias perception towards near
disparities.
space
far disparity least weight
near disparity some weight
zero disparity most weight
time
86
Stroboscopic Pulfrich effect
Physiologically, this bias does not need joint
encoding (i.e., tilted receptive fields).
space
time
87
Stroboscopic Pulfrich effect
This receptive field is tuned to NEAR disparity.
It will respond to the strobe stimulus
space
time
88
Stroboscopic Pulfrich effect
But a cell tuned to FAR disparity will not
respond
space
time
89
These realistic receptive fields account for the
imbalance between the activity near and far
sensors.
space
time
90
Modeling

• If we include what we know about how cortical
  cells respond over time, we can predict the
  amount of depth we expect subjects to perceive in
  the stroboscopic Pulfrich effect.

Read Cumming 2005 Journal of Vision 5 417
91
Stroboscopic Pulfrich effect
Predictions depend on T and ??t.
and on ??t, time over which visual input is
integrated.
T
space
?t
time
92
Stroboscopic Pulfrich effect
Predictions depend on T and ??t.
and on ??t, time over which visual input is
integrated.
T
space
?t
93
perceived depth as a function of interocular delay
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
94
perceived depth as a function of interocular delay
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
95
perceived depth as a function of interocular delay
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
96
perceived depth as a function of interocular delay
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
97
perceived depth as a function of interocular delay
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
98
perceived depth as a function of interocular delay
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
99
perceived depth as a function of interocular delay
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
100
perceived depth as a function of interocular delay
Our model
T/??t 1 T/??t 2 T/??t 3 T/??t 4 T/??t 5
?x/X perceived disparity / strobe spatial period
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
??t/T interocular delay / strobe temporal period
101
Testing the model

• Find out how much depth the interocular delay
  really does cause.
• Add real depth (stereo disparity) in the
  opposite direction
• When we have added enough depth in order to
  cancel out the Pulfrich depth, the pendulum
  appears to swing in a flat plane.

102
back to the psychophysics lab
103
T31ms T62ms T125ms
HN
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
0
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.6
-0.6
-0.6
-0.5
0.5
-0.5
0.5
-0.5
0.5
0
0
0
?x/X perceived disparity / strobe spatial period
BGC
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
0
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.6
-0.6
-0.6
-0.5
0
0.5
-0.5
0
0.5
-0.5
0
0.5
??t/T interocular delay / strobe temporal period
104
Summary

• We do indeed find an S-shaped curve.
• The data supports our model.
• This suggests that joint-encoding is not
  required.
• We conclude that all disparity-tuned cells in V1
  contribute to the Pulfrich illusion.

105
Conclusions

• Neurons initially encode different aspects of the
  stimulus (motion, depth) separately.
• Subsequently, these stimulus attributes are
  unified into a single percept.

106
Consensus ca. 2005

• Key difference between classic Pulfrich effect
  and strobe Pulfrich
• Classic Pulfrich reflects stimulus geometry
• temporal delay ? spatial disparity
• Strobe Pulfrich reflects brain mechanisms
• neurons that encode both motion and disparity
• neuronal basis of Pulfrich effect

107
Our take on it
Basically no

• Key difference between classic Pulfrich effect
  and strobe Pulfrich
• Classic Pulfrich reflects stimulus geometry
• temporal delay ? spatial disparity
• Strobe Pulfrich reflects brain mechanisms
• neurons that encode both motion and disparity
• neuronal basis of Pulfrich effect

108
Our take on it
Basically no

• Key difference between classic Pulfrich effect
  and strobe Pulfrich
• Classic Pulfrich reflects stimulus geometry
• temporal delay ? spatial disparity
• Strobe Pulfrich reflects brain mechanisms
• neurons that encode both motion and disparity
• neuronal basis of Pulfrich effect

• Strobe Pulfrich actually also reflects stimulus
  geometry, plus finite integration time of
  cortical neurons.

109
Modeling
Psychophysics
Physiology
110
Modeling
Psychophysics
the strobe Pulfrich illusion
Physiology
111
the strobe Pulfrich illusion
Modeling
Psychophysics
the strobe Pulfrich illusion
Physiology
112
the strobe Pulfrich illusion
Modeling
Psychophysics
joint encoding of motion and depth?
Physiology
113
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
joint encoding of motion and depth?
Physiology
114
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
joint encoding rarely occurs.
Physiology
115
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
joint encoding rarely occurs.
Physiology
joint encoding rarely occurs
116
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
The illusion can be explained using separate
encoding.
Physiology
joint encoding rarely occurs
117
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
The illusion can be explained using separate
encoding
The illusion can be explained using separate
encoding.
Physiology
joint encoding rarely occurs
118
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
The illusion can be explained using separate
encoding
Prediction how perception should quantitatively
vary with interocular delay.
Physiology
joint encoding rarely occurs
119
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
The illusion can be explained using separate
encoding
How perception varies with interocular delay.
Physiology
joint encoding rarely occurs
120
the strobe Pulfrich illusion
joint encoding of motion and depth?
Modeling
Psychophysics
?
The illusion can be explained using separate
encoding
How perception varies with interocular delay.
How perception varies with interocular delay.
Physiology
joint encoding rarely occurs
121
The End

• Psychphysics, computational modelling and
  electrophysiology all combined to yield a new
  understanding of the neuronal basis for this
  perceptual illusion.

122
(No Transcript)
123
T31ms T62ms T125ms
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
?x/X
0
0
0
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.6
-0.6
-0.6
-0.5
0
0.5
-0.5
0
0.5
-0.5
0
0.5
??t/T interocular delay / strobe temporal period
prediction HN BGC