Editing Attention:

1
Editing Attention
using cinematic edits to affect saccadic eye
movements and visual attention.
Interdisciplinary Tea 27/10/03
Tim Smith tim.smith_at_ed.ac.uk Ph.D.
Supervisors John Lee, Helen Pain, Graeme Ritchie
2
Where did Road Runner go ?
3
Outline

• What is Continuity Editing?
• Cognitive Science and film
• A New Approach
• Outline of Experiment
• Analysis
• Conclusions
• Future Directions

4
Editing

• What is editing?
• the splicing together of strips of film
• Digital or analogue the effect is the same
• a sudden cut from one visual scene to another
• Goal?
• make the transition invisible to the viewer
• How?
• continuity editing

5
Continuity Editing

• Heuristics used by editors to assess the
  perceptual impact of an edit.
• Rules-of-thumb based on experience
• Taught as rules in film school even though
• Based on introspection rather than empirical
  findings
• Abstract and hard to explicitly apply

6
Continuity Editing 180 rule
7
Continuity Editing 180 rule (2)

• Effects of discontinuity cited in editing
  literature
• Crossing the line can confuse the viewer.
• It may also create a jump in the image when an
  unexpected cut occurs
• Impedes narrative comprehension
• Creates unwanted arousal
• all of these are ripe for psychological
  investigation

But why use Cognitive Science to study film?
8
Why use Cognitive Science to study film?

• Cognitive foundations of motion pictures should
  be investigated because
• Motion pictures are Pervasive
• Used for all purposes (e.g. communication,
  entertainment, education, persuasion)
• Feared and loved yet barely understood
• Used as a tool for testing other psychological
  processes
• E.g. Memory, Attention, Emotion, Perception
• Yet the processes involved in the actual
  perception of the film are not understood

9
Previous investigations

• Crossing the line negatively affects memory
• Frith, U. and Robson, J. E. (1975)
• Change blindness persists across continuity edits
• Levin, D.T. and Simons, D.J. (1997)
• Discontinuity is followed by attentional search
• DYdewalle, G., Desmet, G,. and Van Rensbergen,
  J. (1998)

10
Previous investigations conclusions

• The effects of violating the rules of continuity
• can be tested and quantified using psychological
  methodologies.
• Cognitive Film Theory (Hochberg and Brooks, 1978
  Anderson, 1996)
• have many dimensions
• exist at all levels of cognitive processing.
• But,
• level of control of visual information minimal
• Binary test of continuity, no levels

11
Ecological Cognitive Film Theory
(Anderson, 1996)

• We use the same perceptual apparatus and
  behaviours to view artificial visual worlds as we
  do the real world.
• Saccades
• Fixations
• Attentional Orienting
• Visual Masking
• Smooth Pursuit
• If continuity editing is intended to control
  perception and narrative comprehension it must
  first control the visual information extracted
  from the scene.

12
Eye movements and Attention
13
Saccadic Eye Movements

• Most frequent type of eye movement.
• Ballistic rotation of the eye ball
• Targeted using peripheral information
• Goal is to relocate fovea, most sensitive region
  of the retina

Saccadic Eye Movement
Occular Smooth Pursuit
14
Saccadic Eye Movements (2)

• Visual attention is attenuated during saccades
  due to saccadic suppression
• Duration 200ms
• 50-100ms preparation and recovery
• 30ms actual movement
• Target information following the saccade is
  extrapolated back to fill-in the blank.

15
Experiment
16
Main Hypothesis

• Continuity editing attempts to control/accommodate
  visual
• attention so that visual cues of discontinuity
  contained
• within the screen do not make it to the level of
  conscious
• perception.

17
A New Approach

• Explicitly represent rules of continuity
• Create complex dynamic visual scenes under
  experimental conditions
• Operationalise editing factors
• Isolate and control individual factors during
  presentation in an attempt to examine their
  effect.
• Measure attention and memory to map the
  development from vision to perception.

18
Where to begin?

• Directional action continuity
• 180 rule
• Level of eye movements and attention
• Generate stimuli in controlled fashion
• Simplify visual detail to control for extraneous
  effects on attention
• Before perception, first look at continuity of
  attention
• Lateral Stitch

19
Operationalising the edit Trigger

• 3 primary factors
• Trigger, Entry, Attentional Fade-In Duration
  (AFI)
• Trigger the amount of the principle object that
  has left the screen at the point of edit.
• Measured as a percentage of the principle object
  width in relation to the departing screen edge.
• Dependent on motion of principle object relative
  to screen edge temporal and spatial components

20
Operationalising the edit Entry

• Entry Percent the position the principle object
  relocates to on the screen after an edit.
• Created by a movement of the camera yet manifests
  itself as an relocation of the visual scene in
  the opposite direction.
• Measured as a percentage of the principle object
  width in relation to the incoming screen edge.
• a measure of graphical space

21
Operationalising the edit AFI

• Attentional Fade-In Duration (AFI)
• the time period between the onset of the edit
  and the first occurrence of a significant action
• Measured as number of frames between onset of the
  edit and first significant action.
• Action continues, does not pause
• measures the temporal dimension of attention

22
Suggestions from Continuity Editing

• Trigger an edit should be motivated by the
  principle action and coincide with the point of
  greatest action (Anderson, 1996)
• Entry the 180 rule must be followed i.e. the
  direction of the principle motion on the screen
  should stay the same.
• AFI the principle action should be overlapped by
  about two frames across the edit. (Anderson,
  1996)

23
Analogous Psychological domains

• Trigger and Entry
• Dynamic Visual Occlusion the movement of an
  object behind an opaque surface in 3D space so
  that our view of it is obstructed (Michotte,
  1964)
• Ideal conditions for the preservation of
  existence and spatiotemporal continuity
• Gradual local perspectival occlusion
• Benefits
• preservation of object details
• Occluding edge functions as a cue for attention
  modulation

24
Analogous Psychological domains (2)

• Eye Movements
• Trigger
• Preparation for saccade can only occur if clear
  visual cues for edit onset are provided.
• 100ms attention fade-out prior to eye movement
• Entry
• Distinct peripheral visual target is needed for
  saccade initiation.
• Initiation of smooth pursuit is easier when
  originating from behind an occluder. (Churchland
  et. al., 2003)
• AFI
• Total duration of saccade 200ms
• Period of null attention is dependent on distance
  of eye movement 30ms

25
Main Hypothesis

• The edit which conforms to the continuity
• guidelines will best accommodate the necessary
• eye movements and show the best level of
• attention throughout the course of the visual
• sequence.

• Level of attention measured using reaction time
  task
• accuracy of responses how attention is being
  allocated
• null response rate absence of attention
  at specific location
• speed of responses relative levels of
  attention
• Best performance fast reaction time whilst
  maintaining a high accuracy and low null response
  rate either side of the edit.

26
Experimental Hypotheses

• Full occlusion Trigger will show better relative
  performance on an attention test performed
  immediately before and after the edit.
• Full occlusion Entry condition will show better
  attention than those in which it appears already
  visible on the screen.
• The longer the time period between the last
  significant action prior to the edit and that
  following the edit
• a) the greater attention will be allocated to
  these actions (either side of the edit), and
• b) the shorter the recovery period will be before
  the full level of attention is achieved.

27
Experiment demonstration and description
28
Analysis
What is the effect of Trigger condition on Null
Response rate?
Stats Cue pos 1 100-0 (plt.05,
one-tailed) 100-50 (plt.05, one-tailed) Cue
pos 3 and 4 plt.05 (Kruskal-Wallis)
29
Analysis (2)
What is the effect of Trigger condition on
Reaction Time?

• Repeated-measures ANOVA of mean RTs across all
  trials compared within trigger groups does not
  support this hypothesis. The difference between
  RTs at pos0 and 1 is highly significant at 0
  trigger (F15.662, plt.01), 100 (F10.525,
  plt.05), and the Random trigger (F8.312, plt.05),
  but just nonsignificant at 50 (F5.617, p.56).
  This does not support our hypothesis although by
  looking at the graph we can see that there does
  appear to be a trend in the expected direction
  under 50 and 100 trigger conditions.
• -between trigger groups analysis shows no sig.
  diff. probably due to variance within groups
• - Trend of gradual increase in RTs approaching
  the edit matches expectation but variation within
  each group masks any significant effect.

30
Analysis (3)
What effect does Attentional Fade-In duration
have on Null Response rate?
Cue pos. 1 All AFI, (plt.000) Friedman Cue pos.
3 All AFI, (plt.01) Friedman Cue pos. 4 All
AFI, (plt.01) Friedman Cue pos. 5 All
AFI, (plt.05) Friedman
31
Analysis (4)
What is the effect of Entry on Response Accuracy?
Cue position 1 0-100 (plt.01,
one-tailed) Wilcoxon. Cue pos. 4 All
Entry (plt.01) Friedman Cue pos.
5 100-0 (plt.01, one-tailed) Wilcoxon.
32
Analysis (5)
What is the effect of Entry on Reaction Time?
Cue pos. 1 Missing data Cue pos.
4 0-Random (plt.05) t-test
33
Conclusions

• Hypothesis 1 supported
• Performance is optimum under full-occlusion
  Trigger.
• Hypothesis 3 partially supported
• Performance is better when there is time between
  the edit and the first significant action.
• But
• Edge effects interact with this
• Not enough levels to reveal time course
• Hypothesis 2 is not supported
• Only immediate benefit of occlusion during entry
• Eradicated at 125ms after the edit by edge effect
• Therefore, ideal edit
• Trigger 100, Entry100, AFI 3 frames
  (125ms)

34
Future plans

• Ideal entry location?
• Breaking down Attentional Fade-In duration
• Spatial overlap Vs. temporal overlap
• Chronostasis
• Spatial compression
• Need richer visual scenes with
• Temporal and
• Spatial dimensions

35
Where did Road Runner go ?
36
What effect does Attentional Fade-In duration
have on Reaction times?
37
Edge effect
38
(No Transcript)