Attention

1
Attention Perception
2
Outline
1. The problem of visual attention 2. Behavioural
studies of the attentional modulation of
visual aftereffects in humans 3. Mechanisms of
selective visual attention in the primate
brain - single cell studies 4. Imaging studies
of the attentional modulation of
motion processing in the human brain
3
1. The problem of visual attention
4
Three basic phenomena define the problem of
visual attention
1. Limited capacity for processing information -
at any given time only a small amount of the
information available on the retina can be
processed and used in the control of
behaviour 2. Selectivity - the ability to
filter out unwanted information. 3. Integration
- the combining of several object properties such
as colour, shape, location, orientation, and so
on - binding.
Desimone Duncan (1995)
Desimone Duncan (1995)
5
Limited Processing Capacity and Selectivity in
Human Vision
P
N
X
C
?
E
M
B
D
a
b
c
Subjects are shown the displays briefly and asked
to report only the black letters. Limited
capacity is shown by reduced accuracy as the
number of targets is increased (compare b and c).
Selectivity is shown by negligible impact of the
number of distractors - non-targets (compare a
and c).
Desimone Duncan (1995)
6
Search is much easier when distractors share just
one feature with the target (large, vertical
white bar) rather than two - i.e. (b) is easier
than (a) because distractors are white or
vertical or large. In (a), they are large and
vertical or white and vertical or large and
white
(a)
(b)
7
2. Behavioural studies of the attentional
modulation of visual aftereffects in humans
8
Modulation of the Motion Aftereffect by Selective
Attention
Experiment 1 motion adaptation with central
attentional window containing character stream
for alphanumeric discrimination task. Subjects
responded to numerals (4Hz. p1/8). Textured
background moved in one of 4 directions.
Chaudhuri (1990)
9
Experiment 1 Mean Aftereffect Durations for each
of 5 observers
2-tailed t-test S1-4 p lt 0.001 S5 p lt 0.005
Chaudhuri (1990)
10
Experiment 2 Mean Aftereffect Durations with the
fixation point displaced by 2º from the locus of
attention
2-tailed t-test S1-4 p lt 0.001 S5 N/S
Chaudhuri (1990)
11
Experiment 3 Mean Aftereffect Durations when the
attentional task was a colour discrimination of
the background. Subjects responded to a red
background (4Hz. p1/4).
2-tailed t-test S1-5 N/S
Chaudhuri (1990)
12
Figural aftereffect - after Kohler Wallach
(1944) adapting to black squares while fixating
cross subsequently causes white test squares to
appear displaced -
Yeh, Chen, De Valois De Valois (1996)
13
Yeh et al. (1996) - repeated Chaudhuris MAE
result with the FAE ..
14
.. and got no attentional effect when the
letters and digits were inside the Gabor patches
.. much like Chaudhuri got no effect when the
attentional colour coincided with the
moving texture.
15
or when subjects judged whether gratings in
Gabor patch inducers had the same or
different orientations.
16
Attentional modulation of adaptation to
transparent motion
Adaptation ? no adaptation ? two moving ? one
moving, one static
So attentional effect is large but less
than unidirectional effect
Lankheet Verstraten (1995)
17
3. Mechanisms of selective visual attention in
the primate brain - single cell studies
18
What (ventral) pathway
Where (dorsal) pathway
19
The Ventral Stream V1 ? V4 ? TEO ? Inferior
temporal cortex (TE) ? Prefrontal cortex The
What pathway Proceeding up the hierarchy,
neuronal properties change in 2 ways 1. the
complexity of visual processing increases (V1
local spatio- temporal energy IT global object
features such as shape) 2. receptive field size
increases (V1 0.2 TE 25)
Desimone Duncan (1995)
20
Selective attention gates visual processing in
the extrastriate cortex
The monkey was trained to attend to the stimuli
at one location but ignore the stimuli at the
other. The task used to focus the animals
attention was a modified version of a
match-to-sample task.
Moran Desimone (1985)
21
Attention index AI response to stimulus when
ignored response to stimulus when attended so
AI lt 1.0 (dashed line) indicate reduced response
when stimulus ignored
When attention is directed to one of two stimuli
in V4 (or the IT cortex), the effect of the
unattended stimulus is attenuated, almost as
if the receptive field has contracted around the
attended stimulus.
Moran Desimone (1985)
22
What happens when the non-preferred object is
outside the receptive field, so that there is no
competition in the RF?
Moran Desimone (1985)
23
when attention is directed outside a
receptive field, the receptive field appears to
be unaffected.
since the firing rates of cells were the same
regardless of whether attention was directed
inside or outside the receptive field, we
can conclude that attention does not serve to
enhance responses to attended stimuli.
attention gates visual processing by filtering
out irrelevant information from within the
receptive fields of single extrastriate neurons.
Moran Desimone (1985)
24
Vidyasagar (1998)
25
Transcranial magnetic stimulation of human V5
(Walsh et al., 1998)
6 visual search tasks (a) motion pop-out (b)
form pop-out (c) colour- form conjunction (d)
motion irrelevant (e) motion-form conjunction
with moving target (f) motion-form conjunction
with static target.
26
Walsh et al., 1998
motion pop-out
form pop-out
colour x form conjunction
motion -form conjunction stationary target
motion -form conjunction moving target
motion irrelevant
27
TMS applied over human cortical area V5
disproportionately impairs performance on visual
search tasks which require attention to
motion and enhances performance when attention is
directed to attributes other then motion
(Walsh et al., Proc. R. Soc. Lond. B, 265, 1998,
537543.)
28
We conclude that visual area V5 is important
for attention to motion. Additionally, we suggest
that visual areas compete for processing
resources and that disruption of V5 allows
greater resources to be directed to areas
attending to stimulus elements other than motion.
(Walsh et al., 1998)
29
Increased attention enhances behavioural
neuronal performance
Match-to-sample task Easy condition - matching
test stimuli differ by 90? of orientation or 77
nm. in wavelength. Hard condition - 22.5 ? or 19
nm.
Spitzer et al. (1988)
30
Behavioural
The overall performance of the animals in the
easy and difficult conditions was 93 and 73
correct, respectively, suggesting that
the difficult condition was, indeed, more
difficult. A difference in error rate alone,
however, does not prove that the animals actually
processed the stimuli differently in the two
conditions. To examine this question, difficult
non-matching probe trials were inserted randomly
on 6 of the trials within the easy task. The
difficult probe trials were performed with far
more errors when they were presented within the
easy condition (48 correct) than when they were
presented within the difficult condition (78
correct). A signal detection analysis of these
results indicated that in the difficult condition
the discriminability of the stimuli increased (d
2.11 vs. 1.7), that is, the animals internal
representations of the stimuli were
better separated independent of the criterion
used to discriminate them.
Spitzer et al. (1988)
31
Neuronal
Of 98 neurons recorded from area V4, 81
responded more strongly to the optimal stimulus
when it was presented in the difficult condition.
Spitzer et al. (1988)
32
There are 2 possible explanations for the
improvement in neuronal responses in the
difficult condition 1. General arousal, which
might improve the responsiveness of all cells in
V4 2. A restricted effect only on the cells
whose receptive fields contained the stimuli the
animal was attending. To decide between these
possibilities, 41 of the same cells were
tested in another condition the task stimuli
were placed outside of the receptive field of
the recorded neuron, while irrelevant stimuli
were presented inside the receptive field. When
the attended stimuli were placed outside the
receptive field of the neuron, the difficult task
no longer caused any overall enhancement of
responses to the stimuli located inside the
field. These results are inconsistent with the
explanation of general arousal.
Spitzer et al. (1988)
33
Effects of object selection on responses of cells
in the IT cortex
time of saccade indicating monkeys response
Saccadic latency was 300 ms. However, 90 ms
before the eye movement, responses diverged as if
attention had shrunk around the chosen target
even though the target was not yet present.
Chelazzi et al. (1993)
34
(No Transcript)
35
Attentional modulation of visual motion
processing in cortical areas MT and MST
1. Monkey fixates cross 2. Stationary dot
appears 3. Animal responds to first dot by
pushing lever, which causes second dot to
appear 4. Dots start to move back and forth
at the same speed - monkey must track first
dot attentionally, and respond when this
dot changes speed. (Fixation must be
maintained on cross throughout.)
Treue Maunsell (1996)
36
Effects of attention on responses of MT/MST
neurons
MT
65 MT 21 MST 3 either MT or MST
Treue Maunsell (1996)
37
Two dots appear inside receptive field
Any of the three dots could be the target to be
tracked ...
Treue Maunsell (1996)
38
Responses with two dots inside the receptive field
MT
44 MT 16 MST 3 either MT or MST
Treue Maunsell (1996)
39
Results
Experiment 1 most cells responded more strongly
when the stimulus inside
the receptive field was the target.
Experiment 2 the responses of most cells
depended greatly on which
dot was the target. When either of the dots in
the receptive field was attended to, the cell
responded most strongly when that dot moved in
the cells preferred direction. When attention
was directed outside the receptive field, cells
maintained a steady level of activity and were
unaffected by motion direction in the RF even
though nothing in the display had changed, only
the locus of attention.
Treue Maunsell (1996)
. but see Seidman Newsome (1999)
40
4. Imaging studies of the attentional modulation
of motion processing in the human brain
41
Attentional modulation of neural processing of
shape, colour velocity in humans
PET study- Subjects discriminated different
attributes of the same set of visual stimuli
which varied 1 jnd in shape, colour or
velocity Divided attention scan minus selective
attention scan to colour, shape or velocity - d'
gt when attention selective
Attention enhanced activity of extrastriate
regions specialized for processing information
related to the selected attribute
Corbetta et al. (1990)
42
Voluntary attention modulates fMRI activity in
human MT-MST
How does voluntary attention to one attribute
of a visual stimulus affect the neural
processing of that stimulus?
OCraven et al. (1997)
43
Comparison of stimulus- driven
attention- driven effects
The attentional effect is much smaller than the
sensory effect. The stationary dot baseline
change (0) increased to 1.14 in attend
stationary dots condition and to 1.51 in
attend moving dots condition. The attentional
effect was 24-32 (.37/1.51 or 1.14) of the
sensory effect (depending on which moving dot
reference is used) but P lt 0.001.
OCraven et al. (1997)
44
Can irrelevant distractors be excluded from
perception?
Lavie Tsal (1994) proposed that capacity for
perception is limited, but that within those
limits perception proceeds automatically. Thus,
although we may not perceive everything, we are
unable to stop perceiving whatever we can. The
extent to which a target task exhausts available
capacity thus determines the extent to which
irrelevant distracting stimuli will
be processed. Target task linguistic judgements
on single words either LOW load - respond when
word is printed in uppercase letters or HIGH load
- respond when a word is bisyllabic. Distractor
task radial visual motion.
Rees et al. (1997)
45
fMRI
V5 response to ignored stationary dots in
periphery and to moving dots
intentions to avoid irrelevant distractors are
not always sufficient for ignoring them. As long
as the target task imposes only a low load
on attention, irrelevant stimuli such as motion
will still be perceived. However, this
irrelevant perception is strongly reduced if the
load of the unrelated task is increased.
Rees et al. (1997)
46
Awareness, Attention and Consciousness II
47
From Rees, Kreiman Koch (2002), adapted
from Felleman Van Essen (1991)
48
From Blake Logothetis (2002) (d) is an
excellent example of monocular rivalry -
cover one eye and fixate it for about 15 sec
49
Ngo, Miller, Liu Pettigrew Current Biology, 10,
2000 , R134-R136
50
From Blake Logothetis (2002) (a,b) Context
effect - the red grating dominates for longer in
(a) than in (b) (c) another example of rivalry
between patterns, not eyes (d) From Wilson, Blake
Lee (2001) When these rival the rivalry
travels in waves around the annulus, slower in
the radial grating and faster in the periphery
51
Another example of pattern rather than eye
rivalry from Kovacs. Papathomas, Yang Feher
(1996) Eye rivalry does occur in rows 2 and 4
but about 40 of the time observers report the
whole monkey or just text and all-green or
all-red dots
52
When rivalry occurs between a house and face,
fMRI signals are larger in the fusiform face area
(FFA) if the face is dominant but larger in the
perihippocampal place area (PPA) when the house
is dominant - From Blake Logothetis (2002).
53
Haxby et al (Science, 2002, 2425-2430
Subjects had to detect object repetition but
with different viewpoints. Correlations were
calculated between even and odd runs for same vs
different objects
54
Correlations were high and positive for same
objects, whether the analysis included the
top-responding area, e.g. FFA for faces, - red
and dark blue - or not - orange and light blue.
The similarity of the correlations with and
without the top-responding area indicates that
coding of objects and faces in extrastriate
cortex is highly distributed and overlapping
55
Grill-Spector The neural basis for object
perception Current Opinion in Neurobiology, 13,
2003, 159-166
a, b. Similar areas for faces vs. textures and
4 legged animals vs. textures c,d. Similar
areas for scenes and houses vs. textures
56
T
Grill-Spector (2003)
57
Grill-Spector (2003)
58
Kanwisher Wojciulik Nature Reviews
Neuroscience, 1, 2000 91-100
Similar brain areas light up using fMRI when
subjects see an image and when they expect it -
increase in baseline firing rate to prime
relevant neurones?
59
Kastner Ungerleider, Ann. Rev. Neurosci., 23,
2000, 315-341 present evidence that top-down
biasing signals due to visual attention affect
neural processing in several ways. These include
...enhancement of neural responses to an attended
stimulus the filtering of unwanted information
by counteracting the suppression induced by
nearby distractors the biasing of signals in
favor of an attended location by increases of
baseline activity in the absence of visual
stimulation and the increase of stimulus
salience by enhancing the neurons sensitivity to
stimulus contrast (p.321).
60
fMRI signals in fusiform face area (FFA)
increase when Ss imagine a face and in
parahippocampal place area (PPA) when they
imagine a house
Kanwisher Wojciulik Nature Reviews
Neuroscience, 1, 2000 91-100
61
From Rees, Kreiman Koch (2002) V1 activity
does not correlate with awareness fMRI signal
is the same whether a left visual field stimulus
is seen or extinguished
62
Rees, Current Opinion in Neurobiology, 11,
2001,150-156 Same fMRI signal change in V1 and
extrastriate cortex in right parietal lobe
patient whether a left field stimulus is seen
(blue) or extinguished (red) - hence, activity in
these areas is not sufficient for awareness
following the damage
63
There is now abundant evidence that conscious
awareness, rivalry, multistable
perception, working memory and attention all
involve networks linking superior parietal
cortex (A), prefrontal cortex (B) and lateral
ventral cortex (C).
A
A
B
B
C
C
Rees, 2001
64
From Rees, Kreiman Koch (2002) A similar
picture to the previous slide but with data from
5 studies on areas involved in conscious
awareness
65

• Similarity of areas involved in stable rivalry of
  faces
• and a face working memory task (Rees, 2001).
• Activity in parietal cortex is found only when
• transitions occur between rivalrous stimuli.

66
Beck, Rees, Frith Lavie, Nature Neuroscience,
4, 2001, 645 - 650
Subjects fixate cross and Try to detect the
letter X and any change in faces or places.
Letters were small so eye movement would render
them illegible. Difficulty was varied between
subjects to roughly equate detections (48
faces 53 places) and misses.
67
Beck, Rees, Frith Lavie, Nature Neuroscience,
4, 2001,645 - 650
Areas where activity was greater for detections
than misses. Notice distinct ventral regions
for faces and places (a) but similar frontal and
dorsal regions (b).
68
Beck, Rees, Frith Lavie, Nature Neuroscience,
4, 2001, 645 - 650 (a) areas where detected gt
undetected (b) areas where undetected gt no
stimulus. Areas in (a) and (b) are different.
Ventral activity in (a) and (b) but no parietal
(dorsal) or frontal in (b). Thus, parietal and
frontal activity may be necessary for conscious
awareness
69
Blaser, Sperling Lu, Proc. Natl. Acad. Sci.,
96, 1999, 11681-11686.

• 5 sequential frames when
• green (G) contrast gt red (R) -
• see b. below. Seen motion is to
• the left (arrow). If R gt G, seen
• motion is to the right.

c. 5 sequential frames when G R - see d. below.
Seen motion is left if subjects are instructed
to attend to G to the right if they are
instructed to attend to R (arrows).
70
Blaser, Sperling Lu, Proc. Natl. Acad. Sci.,
96, 1999, 11681-11686.
Motion was ambiguous when RG - no effective
R advantage (black curve). Attend R effective
29 and 25 R advantage in two left observers
(red curves). Attend G effective 46 and 26 G
advantage in two left observers (green
curves). Thus, where R advantage 0, red curves
show gt50 R direction judgements and lt 50 for
green curves.
71
Many studies indirectly suggest attention
makes stimuli appear stronger but most are open
to other possible explanations e.g. memory
effects, eye movements etc.
Carrasco, Ling Read (Nature Neuroscience, 7,
2004, 308-313) set out to show that covert
attention (i.e attending to but not looking
directly at) actually increases perceived
contrast.
They used a transient attention paradigm
abrupt, salient peripheral stimuli reflexively
capture attention for 120 msec
72
Carrasco, Ling Read (Nature Neuroscience, 7,
2004, 308-313)
73
Sole difference between experimental and
control conditions ISI increased from 53 msec
to 500 msec - too long for transient attention
to persist
74
Carrasco, Ling Read (Nature Neuroscience, 7,
2004, 308-313)
Same result with higher contrast
stimuli standard contrast increased from 6 to
22
75
Carrasco et al 2004
Demonstration of attentional contrast
enhancement Fixating black dot and attending to
left, lower contrast grating makes left and
right gratings appear equal in contrast
76
Carrasco et al, 2004
Control Subjects report grating
with apparently lower contrast but attention
effect is the same
Note error Y-axis should say lt not gt
77
de Fockert et al., Science, 291, 2001, 1083-1806
Working memory needed to use attention to filter
out distractors, which produce more brain
activity under an unrelated concurrent high-
than low- load task - i.e. insufficient resources
to filter out
distractor
78
The idea has been mooted that V1 processing is
necessary for conscious awareness.
Pascual-Leone Walsh, Science, 292, 2001,
510-512 applied transcranial magnetic stimulation
(TMS) to V1 and V5. TMS disrupts activity in
cortical areas but also may induce perceived
phosphenes (flashes of light).
V1-induced phosphenes are stationary, whereas
V5- induced phosphenes move.
79
TMS was applied to V5 at the intensity that
reliably induced a phosphene and to V1 at 20
below threshold, in the same area of the visual
field for both V1 and V5. Subjects rated motion
of V5 phosphenes 1 motion same as V5 alone 2
moving but not as clearly 3 stationary 4
no phosphene (4 point scale).
The critical variable was the time difference
between application of TMS to V1 and V5. This
varied between -80 msec and 80 msec, in 10 msec
steps. Negative differences indicate that V1 was
applied before V5, positive differences that V5
was applied before V1.
80
Control experiment squares mean rating near 1
- i.e. moving - when paired TMS both to V5 only.
Circles V5 phosphenes rated near 3 - not moving
- when V1 was stimulated 10-45 msec after V5.
This suggests that awareness of extrastriate
processing requires re-entry - feedback - into
V1.
81
Silvanto et al (2005) subthreshold TMS to V5
followed 40 msec later by subthreshold V1 TMS
no phosphenes but
. V5 subthreshold then V1 suprathreshold moving
phosphenes and, at -40 msec, percept like a
mixture of V5 and V1 phosphenes
Shows that awareness via V1 arises from
extrastriate feedback
82
Monkeys were presented with just a
background texture (a) or a texture-defined square
in one of 3 positions (b)
When they detected a square they were trained to
make an eye movement to it
83
Extra response to square starts about 80 msec
after initial V1 response suggesting feedback
from extrastriate cortex and re-entry to V1
mediates detection. Note no extra response when
square is present but undetected
Super, H., Spekreijse, H. Lamme, V. A. Nature
Neurosci. 4, 304310 (2001).
84
The Role of V1
V1 is necessary for our conscious visual
awareness ...
... it is nevertheless not sufficient for
awareness
V1 needs to be functioning at least partially in
order that we are aware of visual stimulation ...
85
but we are not aware of processing which occurs
in V1 itself...
... only of processing in higher, extrastriate
areas to which V1 feeds ...
... and which, in turn, feed back into V1 to
enable conscious vision
86
Moutoussis Zeki, Proc. Nat. Acad. Sci., 99,
2002, 9527-9532
Opposite Different isoluminant dichoptic
stimuli produce the same percept (yellow)
Same Same isoluminant dichoptic stimuli produce
different percepts (square, house or face)
Opposite stimuli cancel rather than rival because
they are flashed very briefly. Graph shows that
subjects almost always perceive same but hardly
ever see opposite stimuli seen-opposites were
removed from analysis
87
The same brain areas are active when stimuli are
perceived (same) than when not perceived
(opposite), compared with square control. Just
less activation when not perceived. Hence,
activation of face/house areas is not sufficient
for perception.
88
Same stimulus/ different percepts Difference
between same and opposite for houses and
faces shows that activation is greater in house
and face areas when the stimulus is perceived
than when it is not