Dynamic attention and predictive tracking

1
Dynamic attention and predictive tracking
Lomonosov Moscow State University Cognitive
Seminar, 6/10/2004

• Todd S. Horowitz
• Visual Attention Laboratory
• Brigham Womens Hospital
• Harvard Medical School

2
lab photo
David Fencsik
Randy Birnkrant
Jeremy Wolfe
Linda Tran (not pictured)
3
Multi-element visual tracking task (MVT)

• Devised by Pylyshyn Storm (1988)
• Method for studying attention to dynamic objects

4
Multi-element visual tracking task (MVT)

• Present several (8-10) identical objects
• Cue a subset (4-5) as targets
• All objects move independently for several
  seconds
• Observers asked to indicate which objects were
  cued

5
Demo
demo
mvt4
6
Interesting facts about MVT

• Can track 4-5 objects (Pylyshyn Storm, 1988)
• Tracking survives occlusion (Scholl Pylyshyn,
  1999)
• Involves parietal cortex (Culham, et al, 1998)
• Clues to objecthood - Scholl

7
Accounts of MVT performance

• FINSTs (Pylyshyn, 1989)
• Virtual polygons (Yantis, 1992)
• Object files (Kahneman Treisman, 1984)
• Object-based attention

8
These are all (partially) wrong

• FINSTs (Pylyshyn, 1989)
• Virtual polygons (Yantis, 1992)
• Object files (Kahneman Treisman, 1984)
• Object-based attention

9
Common assumptions

• Low level (1st order) motion system updates
  higher-level representation
• FINST
• Object file
• Virtual polygon
• Continuous computation in the present

10
Overview

• MVT and attention
• Tracking across the gap
• Tracking trajectories

11
MVT and attention

• Clearly a limited-capacity resource
• Attentional priority to tracked items (Sears
  Pylyshyn)
• Hypothesis MVT is mutually exclusive with other
  attentional tasks

George Alvarez, Helga Arsenio, Jennifer DiMase,
Jeremy Wolfe
12
MVT and attention

• Clearly a limited-capacity resource
• Attentional priority to tracked items (Sears
  Pylyshyn)
• Hypothesis MVT is mutually exclusive with visual
  search

13
MVT and attention

• Clearly a limited-capacity resource
• Attentional priority to tracked items (Sears
  Pylyshyn)
• Hypothesis MVT is mutually exclusive with visual
  search
• Method Attentional Operating Characteristic (AOC)

14
AOC Theory
15
General methods - normalization

• Single task 100
• Chance 0
• Dual task performance scaled to distance between
  single task performance and chance

16
General methods - staircases

• Up step (following error) 2 x down step
• Asymptote 66.7 accuracy
• Staircase runs until 20 reversals
• Asymptote computed on last 10 reversals

17
General methods - tracking

• 10 disks
• 5 disks cued
• Speed 9/s

18
AOC Theory
19
AOC reality

• Tasks can interfere at multiple levels
• Interference can occur even when resource of
  interest (here visual attention) is not shared
• How independent are two attention-demanding
  tasks which do not share visual attention
  resources?

20
Gold standard tracking vs. tone detection
21
Gold standard method

• Tracking
• Duration 6 s
• Tone duration
• 10 600 Hz tones
• Onset t 1 s
• ITI 400 ms
• Distractor duration 200 ms
• Task target tone longer or shorter?
• Target duration staircased (?31 ms)
• Dual task priority varied

N 10
22
Gold standard AOC
23
Tracking search method

• Tracking
• Duration 5 s
• Search
• 2AFC E vs. N
• Distractors rest of alphabet
• Set size 5
• Duration staircased (mean 156 ms)
• Onset 2 s

N 9
24
Tracking search method
25
Tracking search AOC
26
Tracking search AOC
27
Does tracked status matter?
L
L
L
28
method

• Tracking
• Duration 3 s
• Search
• 2AFC left- or right-pointing T
• Distractors rotated Ls
• Set size 5
• Duration staircased (mean 218 ms)
• Onset 1 s

N 9
29
search inside tracked set
L
L
T
L
L
30
search outside tracked set
L
L
L
T
L
L
31
(No Transcript)
32
inside vs. outside AOC
33
Does spatial separation matter?
P
E
H
F
V
34
method

• Tracking
• Duration 5 s
• Search
• 2AFC E vs. N
• Distractors rest of alphabet
• Set size 5
• Duration 200 ms
• Onset 2 s

N 9
35
spatial separation AOC
36
search v track summary
37
MVT and search

• Clearly not mutually exclusive
• Not pure independence
• Close to gold standard
• MVT and search use independent resources?

38
Two explanations

• Separate attention mechanisms
• Time sharing

39
Predictions of time sharing hypothesis

• Should be able to leave tracking task for
  significant periods with no loss of performance
• Should be able to do something in that interval

40
Track across the gap method
41
Track across the gap method

• Track 4 of 8 disks
• Speed 6/s
• Blank interval onset 1, 2, or 3 s
• Trajectory variability 0, 15, 30, or 45
  every 20 ms
• Blank interval duration staircased (dv)
• N 11

42
track across the gap asymptotes
43
Predictions of time sharing hypothesis

• Should be able to leave tracking task for
  significant periods with no loss of performance
  (see also Yin Thornton, 1999) - confirmed
• Should be able to do something (e.g. search) in
  that interval

44
search during gap method

• AOC method
• Tracking task same as before
• Search task in blank interval
• Target rotated T
• Distractors rotated Ls
• Set size 8
• 4AFC Report orientation of T
• Duration of search task staircased (326 ms)

45
search during gap AOC
46
(No Transcript)
47
Predictions of time sharing hypothesis

• Should be able to leave tracking task for
  significant periods of time with no loss of
  performance (see also Yin Thornton, 1999) -
  confirmed
• Should be able to do something (e.g. search) in
  that interval - confirmed

48
Summary

• MVT and visual search can be performed
  independently in the same trial
• May support independent visual attention
  mechanisms
• May support time-sharing

49
Summary

• Tracking across the gap data support time sharing
• Tracking across the gap data raise new questions

50
What is the mechanism?

• Not a continuous computation in the present
• Not first order motion mechanisms
• Not apparent motion

Randall Birnkrant, Jennifer DiMase, Sarah
Klieger, Linda Tran, Jeremy Wolfe
51
None of these theories fit

• FINSTs (Pylyshyn, 1989)
• Virtual polygons (Yantis, 1992)
• Object files (Kahneman Treisman, 1984)

52
What is the mechanism?

• Some sort of amodal perception? (e.g. tracking
  behind occluders, Scholl Pylyshyn, 1999)
• but there are no occlusion cues!

53
(No Transcript)
54
(No Transcript)
55
Scholl Pylyshyn, 1999
56
Maybe the gap is just an impoverished occlusion
stimulus

• No occlusion/disocclusion cues
• Synchronous disappearance

57
Predictions of impoverished occlusion hypothesis

• Occlusion cues will improve performance
• Asynchronous disappearance will improve
  performance

58
Method

• Track for 5 s
• Speed 12/s
• Track 4 of 10 disks
• Independent variables (blocked)
• Gap duration107 ms, 307 ms, 507 ms
• Occlusion cues absent, present
• Disappearances synchronous, asynchronous
• N 15

59
synchronous disappearance
60
synchronous disappearance occlusion
61
Occlusion/Disocclusion
62
asynchronous disappearance
63
asynchronous disappearance occlusion
64
comparing cue types
65
Occlusion hypothesis fails

• Occlusion cues dont help
• Asynchronous disappearance doesnt help

66
Method

• Track for 5 s
• Speed 12/s
• Synchronous condition only
• Independent variables (blocked)
• Gap duration107 ms, 307 ms, 507 ms
• Occlusion cues absent, present
• Track 4, 5, or 6 of 10 disks
• N 11

67
comparing cue types
68
Occlusion hypothesis fails

• Occlusion cues dont help
• Occlusion cues can actually harm performance
• Asynchronous disappearance doesnt help

69
What is the mechanism?

• Not a continuous computation in the present
• Not first order motion mechanisms
• Not apparent motion
• Not amodal perception (occlusion)

70
How do we reacquire targets?

• remember last location (backward)
• store trajectory (forward)

David Fencsik, Sarah Klieger, Jeremy Wolfe
71
location-matching account
Memorized pre-gap target location.
Nearest to memorized location identified as
target.
First Post-Gap Frame
72
trajectory-matching account
Memorized pre-gap target trajectory.
On target trajectory identified as target.
First Post-Gap Frame
73
Shifting post-gap location
74
shifting post-gap location predictions
75
Shifting post-gap location methods

• track for 5 s
• speed 8/s
• track 5 of 10 disks
• gap duration 300 ms
• post-gap location condition blocked
• stimuli continue to move after gap

76
shifting post-gap location
77
Location vs. trajectory-matching

• support for location-matching
• see also Keane Pylyshyn 2003 2004
• but advantage for -1 is suspicious

78
Location vs. trajectory-matching



79
shift stop methods

• track for 4-6 s
• speed 9/s
• track 2 or 5 of 10 disks
• gap duration 300 ms
• post-gap location condition blocked
• stimuli stop after gap

80
moving vs. static after gap
81
moving vs. static after gap
82
2 vs. 5 targets
83
Location vs. trajectory-matching

• support for location-matching
• However...
• conditions are blocked
• observers might see their task not as tracking
  across the gap, but learning which condition
  theyre in
• might not tell us about normal target recovery

84
Location vs. trajectory-matching

• can subjects use trajectory information?
• always have items move during gap
• vary whether trajectory information is available
  or not

85
moving condition
86
static condition
87
manipulate pre-gap information methods

• track for 4 s
• speed 9/s
• track 1 to 4 of 10 disks
• gap duration 300 ms

88
manipulate pre-gap information
89
manipulate pre-gap information
90
Location vs. trajectory-matching

• observers can use trajectory information
• unlimited (or at least gt 4) capacity for
  locations
• smaller (1 or 2) capacity for trajectories

91
Conclusions

• Flexible attention system allows rapid switching
  between MVT and other attention-demanding tasks
• Some representation allows recovery of tracked
  targets after 300-400 ms gaps
• This representation includes location and
  trajectory information

92
Speculation

• MVT reveals two mechanisms, rather than just one
• Frequently (but perhaps not continuously) updated
  location store
• Attention to trajectories