An Attentional Theory of Continuity Editing

1
An Attentional Theory of Continuity Editing

• Tim J. Smith
• 21/07/06
• tim.smith_at_ed.ac.uk
• http//homepages.inf.ed.ac.uk/s9732397/

2

• B.L.U.F. (Bottom Line Up Front)
• Continuity existence constancy
• Continuity editing rules requirements for
  existence constancy

3

• What is continuity?
• Continuity Making a smooth cut means joining
  two shots in such a way that the transition does
  not create a noticeable jerk and the spectators
  illusion of seeing continuous motion is not
  interrupted. (Reisz Millar, 1953)

• Existence Constancy is the experience that
  objects persist through space and time despite
  the fact that their presence in the visual field
  may be discontinuous (Butterworth, 1991).

4

• Continuity existence constancy?
• Show that
• the requirements are the same
• by adhering to those requirements a cut can be
  created that has existence constancy.
• ? Create a cut that has existence constancy.

5
Existence Constancy does not require full
continuity We sometimes perceive objects within
the world as being continuous even though they
change ? Change Blindness

• Change Blindness
• Selective and sparse representations
• We rely on the world to tell us when something
  has changed (Visual transients).
• We have to expect a change to detect without
  transients.
• Otherwise we assume existence constancy.

Simons Levin, 1998
6
(No Transcript)
7

• Change Captures Attention
• Visual transients involuntarily capture a
  viewers attention.
• flicker, motion, dis/appearances, colour/
  brightness changes.
• To hide a change
• mask the transients, or
• shift attention elsewhere
• ? misdirection.
• Cause overt Attentional shifts (Saccades)
• ? 200ms perceptual hole
• What changes can we use to hide a cut?

2
8

• Using natural attentional cues
• Most dominant cue is motion.
• Wait until the woman makes some kind of
  movement and then cut during the motion.

9

• Match Action Editing
• ? We have just devised the match action editing
  rule.

the cutter should look for some movement by the
actor who holds the viewers attention, and he
should use that movement to trigger his cut from
one scene to another. A broad action, will offer
the easier cut, but even a slight movement of
some part of the players body can serve to
initiate a cut which will be smooth, or
invisible. The important consideration here is
that there be just enough movement to catch the
viewers attention. (Dmytryk, 1986, page
435-436)
10

• Maintaining Existence Constancy
• If the viewer notices the result of the cut they
  may cease to perceive existence constancy.
• ? Need to satisfy perceptual expectations.
• Perceptual Expectations
• Spatiotemporal
• When and Where an object is expected.
• Stored as an attentional index.
• Object
• Appearance and Semantic details.
• Stored in an object file.

Existence constancy only requires maintenance of
the attentional index not the object file.
Flombaum Scholl, in press
11

• Matching Spatial expectations
• We create existence constancy across the cut by
  ensuring that the objects are roughly in the same
  screen location after the cut as before.

12

• 180 System
• ? We have just devised the 180 System.
• Rotating the camera around a 180 arc ensures
• directional continuity, and
• actors remain in roughly the same position on the
  screen.
• Deviating from this system creates discontinuity
• the actors are not perceived as having
  existence constancy.

13

• But does this result in existence constancy?

1) Viewers are more aware of discontinuity
cuts 2) Viewers can perform change detection
across match cuts. 3) Only films edited for
continuity are recalled accurately.
Levin Simons, 1997 Williams and Simons, 2000
Existence constancy is perceived across match
cuts
Frith Robson, 1975 Smith Greve, in prep.
d'Ydewalle Vanderbeeken, 1990 Schröder,
1990 Smith, in prep
14

• Conclusions
• Continuity existence constancy
• Continuity editing (CE) rules requirements for
  existence constancy
• By identifying the cognitive foundations of CE
• More precise CE rules can be formalised and
  modelled.
• Filmmakers gain a greater understanding of their
  viewers cinematic experience.
• Psychologists learn more about real-world
  cognition.
• Film theorists are given a whole new way of
  looking at film.

15
Thank you tim.smith_at_ed.ac.uk http//homepages.i
nf.ed.ac.uk/s9732397

• References provided on request.
• Any questions?

16
Levels of Continuity?

• Emotion
• Story
• Rhythm
• Eye trace
• 2D composition
• 3D space of action

Co-dependent
Increasing Importance
All essential for the creation of existence
constancy
(Walter Murch, 2001)
But this list needs empirical validation
17

• Bibliography
• Anderson, J. (1996). The Reality of Illusion An
  Ecological Approach to Cognitive Film Theory.
  Southern. Illinois University Press.
• Block, B. (2001). The Visual Story Seeing
  structure of film, TV, and New Media. MA, USA
  Focal Press.
• Butterworth, G. (1991). Phenomenal Permanence. G.
  Thines, A. Costall, G. Butterworth (Eds.),
  Michotte's experimental phenomenology of
  perception (pp. 117-122). Hillsdale, NJ Lawrence
  Erlbaum.
• Cutting, J. (2004). Perceiving Scene in Film and
  in the World. J. D. Anderson, B. F. Anderson
  (Eds.), Moving image theory ecological
  considerations .
• Dmytryk, E. (1986). On Filmmaking. London, UK
  Focal Press.
• d'Ydewalle, G., Vanderbeeken, M. (1990).
  Perceptual and cognitive processing of editing
  rules in film. R. Groner, G. d'Ydewalle, R.
  Parnham (Eds.), From eye to mind Information
  acquisition in perception, search, and reading.
  (pp. 129-139). Amsterdam Elsevier (North
  Holland).
• Flombaum, J. I., Scholl, B. J. (in press). A
  temporal same-object advantage in the tunnel
  effect Facilitated change detection for
  persisting objects. Journal of Experimental
  Psychology Human Perception Performance.

18

• Reisz, K., Millar, G. (1953). Technique of Film
  Editing. London, UK Focal Press.
• Simons, D. J., Levin, D. T. (1998). Failure to
  detect changes to people during real-world
  interaction. Psychonomic Bulletin and Review, 5,
  644-649.
• Schröder, J. (1990). Die psychologische Realität
  von Prinzipien des Continuity Cinema. G. Schumm,
  H. G. Wulff (Eds.), Film und Psychologie I.
  Kognition-Rezeption-Perzeption (pp. 109-142).
  Münster Maks Publikationen.
• Smith, T. J., Greve, A. (in preparation).
  Editing Memory Investigating the effect of
  continuity on memory.
• Smith, T. J. (in preparation) Classifying
  Continuity The relationship between eye
  movements and continuity in moving-picture
  editing.
• Tan, E. S.-H. (1995). Constraint and Convention
  in Psychological Film Aesthetics On the
  Psychological Basis of Cinematic Conventions. J.
  Müller (Ed.), Towards a pragmatics of the
  audiovisual. (pp. 67-86). Münster Nodus.
• Williams, P., Simons, D. J. (2000). Detecting
  Changes in Novel, Complex, Three-dimensional
  Objects. Visual Cognition, 7(1/2/3), 297-322.
• Frith, U., Robson, J. E. (1975). Perceiving
  the language of films. Perception, 4(1), 97-103.
• Levin, D. T. , Simons, D. J. (1997). Failure to
  detect changes to attended objects in motion
  pictures. Psychonomic Bulletin and Review, 4,
  501-506.

19

• 3 Types of attentional cue
• PULL
• POINT
• PUSH

?
?
20
Eyeline Match

• The viewer, as a rule, will not accept the
  fact of a look until he sees the actors eyes
  focus, or freeze on something off-screen. At
  that point he, too, will look off, following the
  actors gaze. By the time his own eyes have
  refocused, the actors point-of-view (POV) shot
  should occupy the screen. To make the cut, then,
  we fix the frame in which the actors eyes have
  frozen, add three or four frames more to give
  the viewer time to react and move his eyes as he
  follows the actors look, at which point the cut
  is made. (Dmytryk, 1986, page 444)

21
Full continuity
22
Crossing the line (similar to Frith Robson,
1975)
23
Identity discontinuity (similar to Levin
Simons, 1997)
24
(No Transcript)
25
(No Transcript)
26
1 2 3 4 5 6 7 8
27
Hasson, U., et al. (2004). Intersubject
Synchronization of Cortical Activity During
Natural Vision. Science, 303, 1634-1640.
This intersubject correlation was attributable
to (FFA) A component driven by faces, (CoS)
indoor and outdoor scenes, and (a nonspecific
component) driven by what?
?Reflects observers segmenting the event into
meaningful units.
28

• The psychological effects of cuts

29

• Object Files (Kahneman Treisman, 1984 1992)
• Mid-level representation of object-level
  information.
• Focal attention binds features together.
• Sticks to moving object via spatiotemporal
  index.
• Correspondence compares current visual
  information to object file (Ullman, 1979).
• Correspondence only occurs if
• viewer older than 10 months (Xu, 1999) and
• adopt a change detection viewing strategy (Levin
  Simons, 1997) or
• visual transients capture attention (Rensink et
  al., 1997).

Perception appears to define objects more by
spatiotemporal constraints than by their sensory
properties or by their labelled identity
(Kahneman, Treisman, and Gibbs, 1992, pg 177)
30

• Visual Index or FINST (Pylyshyn, 1989)
• FINST Fingers of Instantiation.
• Pre-attentive object identifier.
• Overt attentional shifts are quicker to indexed
  objects.
• Visual Indices can be maintained during occlusion
  if
• Objects disappear from view via gradual occlusion
  (Michotte, 1955 Scholl Pylyshyn, 1999)
• Continuous tracking of object by covert
  attention.
• Satisfy spatiotemporal expectations (Hirsch,
  1982 Flombaum Scholl, in press).
• Maintained object file.
• ? Existence constancy

31
(Xu, 1999)
32

• Matched-Exit/Entrance cuts rare (a subset of 14)
• But cuts during conversations most frequent (31)
• (May, Dean, Barnard, 2003)
• Triangle system

1
2
3

Viewers eyetrace is the same irrespective of
camera angle (Block, 2001).

"The Five C's of Cinematography (Mascelli, 1998)
X

• Social attention cues
• Conversational turn taking, addressing the
  partner, pointing,
• head turns towards partner or object, audio
  bleed, etc.
• Viewers might be anticipating cut (May, Dean,
  Barnard, 2003).

33

• Inter-subject agreement of gaze location
  increases as shot size decreases.
• Reorienting of gaze after cut is quicker and more
  consistent in closer shots.
• Medium Close-Ups (MCU) of face increase
  effectiveness of interpersonal cues.
• Audio bleed saccades by 3rd frame/120ms after
  cut.
• Purely visual cues saccades by 4th frame/160ms
  after cut.

MFS
MCU
CU
34
Silent
Sig. peak
Audio
35

• Reference List
• 1. d'Ydewalle, G., Vanderbeeken, M. (1990).
  Perceptual and cognitive processing of editing
  rules in film. R. Groner, G. d'Ydewalle, R.
  Parnham (Eds.), From eye to mind Information
  acquisition in perception, search, and reading.
  (pp. 129-139). Amsterdam Elsevier (North
  Holland).
• 2. Schröder, J. (1990). Die psychologische
  Realität von Prinzipien des Continuity Cinema. G.
  Schumm, H. G. Wulff (Eds.), Film und
  Psychologie I. Kognition-Rezeption-Perzeption
  (pp. 109-142). Münster Maks Publikationen.
• 3. Frith, U., Robson, J. E. (1975).
  Perceiving the language of films. Perception,
  4(1), 97-103.
• 4. Seddon, M. D. (2003). Investigating the
  effect on attention of action continuity. Notes
  Unpublished MSc. Thesis.
• 5. Schwan, S., Garsoffky, B. (2004). The
  Cognitive Representation of Filmic Event
  Summaries. Applied Cognitive Psychology, 18,
  37-55.
• 6. Kraft, R. N. (1987). Rules and strategies of
  visual narratives. Perceptual and Motor Skills,
  64, 3-14.

36

• 7. Geiger, S., Reeves, B. (1993). The effects
  of scene changes and semantic relatedness on
  attention to television. Communication Research,
  20(155-175).
• 8. Reeves, B., Thorson, E., Rothschild, M.,
  McDonald, D., Hirsch, J., Goldstein, R. (1985).
  Attention to television Intra stimulus effects
  of movement and scene changes on alpha variation
  over time. International Journal of Neuroscience,
  25, 241-255.
• 9. Lang, A., Geiger, S., Strickwerda, M.,
  Sumner, J. (1993). The effects of related and
  unrelated cuts on viewers memory for television
  A limited capacity theory of television viewing.
  Communication Research, 20, 4-29.
• 10. Reisz, K., Millar, G. (1953). Technique of
  Film Editing. London, UK Focal Press.
• 11. Comuntzis, G. M. (1987). Children's
  Comprehension of Changing Viewpoints in Visual
  Presentations. Visual Communication Conference .
• 12. Michotte, A. (1955). Perception and
  cognition. Acta Psychologica, 11, 69-91.
• 13. Butterworth, G. (1991). Phenomenal
  Permanence. G. Thines, A. Costall, G.
  Butterworth (Eds.), Michotte's experimental
  phenomenology of perception (pp. 117-122).
  Hillsdale, NJ Lawrence Erlbaum.

37

• 14. Hirsch, E. (1982). The Concept of Identity.
  Cambridge, MA Harvard University.
• 15. Xu, F. (1999). Object individuation and
  object identity in infancy The role of
  spatiotemporal information, object property
  information and language. Acta Psychologica, 102,
  113-136.
• 16. Levin, D. T. , Simons, D. J. (1997).
  Failure to detect changes to attended objects in
  motion pictures. Psychonomic Bulletin and Review,
  4, 501-506.
• 17. Rensink, R. A., O'Regan, J. K., Clark, J.
  J. (1997). To see or not to see the need for
  attention to perceive changes in scenes.
  Psychological Science, 8, 368-373.
• 18. Kahneman, D. , Treisman, A., Gibbs, B. J.
  (1992). The reviewing of object files
  Object-specific integration of information.
  Cognitive Psychology, 24(2), 175-219.
• 19. Kahneman, D. , Treisman, A. (1984).
  Changing views of attention and automaticity. D.
  A. Davies (Ed.), Varieties of attention . New
  York Academic Press.
• 20. Ullman, S. (1979). The interpretation of
  visual motion. Cambridge, MA MIT Press.
• 21. Scholl, B. J., Pylyshyn, Z. W. (1999).
  Tracking multiple items through occlusion clues
  to visual objecthood. Cognitive Psychology, 38,
  259-290.

38

• 22. Pylyshyn, Z. W. (1989). The role of location
  indexes in spatial perception a sketch of the
  FINST spatial-index model. Cognition, 32, 65-97.
• 23. Flombaum, J. I., Scholl, B. J. (in press).
  A temporal same-object advantage in the tunnel
  effect Facilitated change detection for
  persisting objects. Journal of Experimental
  Psychology Human Perception and Performance.
• 24. Dmytryk, E. (1986). On Filmmaking. London,
  UK Focal Press.
• 25. Langton, S. R. H., Watt, R. J., Bruce, V.
  (2000). Do the eyes have it? Cues to the
  direction of social attention. Trends in
  Cognitive Sciences, 4(2), 50-59.
• 26. Williams, P., Simons, D. J. (2000).
  Detecting Changes in Novel, Complex,
  Three-dimensional Objects. Visual Cognition,
  7(1/2/3), 297-322.
• 27. Lindgren, E. (1948). The Art of the Film.
  London, UK George Allen Unwin Limited.
• 28. Block, R. A. , George, E. J., Reed, M. A.
  (1980). A Watched pot sometimes boils a study of
  duration experience. Acta Psychologica, 46,
  81-94.
• 29. Yarrow, K., Haggard, P., Heal, R., Brown,
  P., Rothwell, J. C. (2001). Illusory
  perceptions of space and time preserve
  cross-saccadic perceptual continuity. Nature,
  414.
• 30. Yarrow, K., Whiteley, L., Rothwell, J. C.,
  Haggard, P. (2005). Spatial consequences of
  bridging the saccadic gap. Vision Research, 46,
  545-555.