HOW ARE COGNITIVE MAPS REPRESENTED IN THE BRAIN? VISUAL VS. PROPOSITIONAL REPRESENTATIONS AND IMPLICATIONS FOR GPS TECHNOLOGY

1
HOW ARE COGNITIVE MAPS REPRESENTED IN THE
BRAIN?VISUAL VS. PROPOSITIONAL REPRESENTATIONS
AND IMPLICATIONS FOR GPS TECHNOLOGY

• Serj Mooradian
• Alison Wheatley
• Becky Wright
• Jane Emerton
• Penelope Dimitrakopoulou

2
INTRODUCTION

• Behaviourism from the 1920s onward, behaviour
  was explained in terms of simple
  stimulus-response mechanisms. Rats learnt mazes
  by linking rewards (stimulus i.e. food) with
  muscle movements (response i.e. running in
  particular directions).
• Tolman (1932) cast doubt on this theory rats in
  mazes swimming versus running.
• The researchers concluded that rats
  construct a cognitive map in their
  brains, and use the internal
  representation to solve mazes.

3
MORE ABOUT COGNITIVE MAPPING

• A cognitive map is a cognitive representation of
  an environment, that is built up whenever a
  person encounters a novel environment (Lynch,
  1960). It can comprise of landmarks, nodes, and
  paths.
• The more experience a person has with an
  environment, the better their cognitive map
  (Jacobson et al 2001).
• The type and content of how this information is
  represented in the brain is still a prominent and
  controversial debate.
• There appears at present to be two main arguments
  those who believe cognitive maps are
  represented pictorially vs. those who believe
  they are represented propositionally.

4
HOW ARE COGNITIVE MAPS REPRESENTED IN THE BRAIN?

• 1. PICTORIAL vs. PROPOSITIONAL
• Type and content of cognitive maps is still being
  debated
• Pictorial Kosslyn
• Depiction of environment
• Implicates visual system in production and
  processing the minds eye
• Propositional Pylyshyn
• Similar to language
• Syntactical representation

5
PROPOSITIONAL vs. PICTORIAL

• North, (TREE, HOUSE)
• South (LAKE, HOUSE)
• -Linguistic description of relationships
• -Discrete symbols
• -Requires a symbols to describe the relationships
• -Follows grammatical rules
• -Abstract symbols
• -Unambiguous
• -Multiple modes of perception

• -More easily accessible
• -Does not require knowledge of language
• -Not discrete symbols
• -Does not require a symbol to describe the
  relationships
• -Concrete symbols
• -Ambiguous
• -Only visual mode of perception

6
EARLY COGNITIVE MAP EXPERIMENTS

• Images cannot be directly observed so study brain
  at functional level
• Kosslyn et al (1978) Mental Image Scanning Tasks
• Method
• Memorise a map of 7 locations
• Focus on a location
• Decide if a second location was on the map
• Picture a black speck moving between the two
  locations
• Results
• Scanning time increased linearly with the
  distance between the two locations
• Conclusion
• It takes longer to scan larger distances compared
  to small if images are depictions
• Images are depictive rather than descriptive and
  have spatial properties

7
EARLY EXPERIMENTS cont.ALTERNATIVE MODEL

• Pylyshyn (1981) Critique of Kosslyns
  Experiments
• Participants asked to see a scene
• Use knowledge about really world to simulate
  events
• Does not tell us about inherent nature of imaging
• Experiment
• Scanning phenomenon vanishes when asked to judge
  direction
• Can switch attention between locations without
  simulating moving
• Distance between locations had no effect on
  response time of direction
• Conclusion
• Unclear which properties are due to the inherent
  nature of the mental imaging system and its
  mechanism
• Or which properties are cognitively penetrable
  by the persons own knowledge and beliefs during
  imagery.

8
NEW TECHNOLOGY GIVES US A WINDOW INTO THE MIND

• Attempts to solve debate via behavioural methods
  had led researchers down a cul-de-sac.
• However. Advances in technology offered a new
  opportunity.
• Opened up a window into the mind.

9
PET (POSITRON EMISSION TOMOGRAPHY)

• ²/3 of activation shared by vision and imagery.
• (Kosslyn, Thompson Alpert, 1997)
• However, PET not very precise.
• rTMS (Repetitive Transcranial Magnetic
  Stimulation)

Applying rTMS to the occipital cortex causes
temporary impairment of visual mental imagery
(Kosslyn et al., 1999).
10
PYLYSHYNS RESPONSE

• Pylyshyn (2002) Possible visual system can be
  involved in both vision and mental imagery but
  without generating a picture-like representation.
• It is important not only that such areas be
  involved in imagery, but also that their
  involvement be of the right sort that the way
  their topographical organization is involved
  reflects the spatial properties of the image
• (Pylyshyn, 2002, p.175)

11
RETINOTOPICALITY

• Area 17

Located in the occipital lobe. Involved in early
visual processing e.g. shape, location. Activity
is distributed according to a functional
space i.e. the spatial properties of the visual
scene are reflected in the spatial patterns of
the brain activity (Kosslyn, 1994).
12
Visual Spatial Spectrum
TOP
RIGHT SIDE
BOTTOM
13
FUNCTIONAL SPACE Different spatial areas within
the visual field reflected onto the retina.
Adapted from Slotnick et al (2005)
result in activation in different primary
visual areas.
14
Q. DOES THE RETINOTOPICAL ACTIVITY FOUND IN
VISION ALSO OCCUR DURING MENTAL IMAGERY?

• Klein et al (2004), Slotnick et al (2005)
• fMRI study (Functional Magnetic Resolution
  Imaging)
• Used simple visual stimuli

Asked participants to both visualise and imagine
the stimuli.
15
Vertical-axis space red green areas
Horizontal-axis space blue yellow areas
Vision
Imagery
Adapted from Slotnick et al (2005)
16
PYLYSHYNS RESPONSE TO KOSSLYN

• Kosslyns findings have not been replicated
• They are inherently biased
• Participants were asked to visualize an image
• By visualizing an image, they are being asked to
  see it and hence their visual cortex must be
  stimulated

17
PYLYSHYNS RESPONSE TO KOSSLYNcont.

• Visual images are caused by a response to light
  on the retina and mental imagery is a top-down
  process from higher cortical processes
• It is unlikely that the same response will occur
  to both types of imagery
• Kosslyn literally pictures in the brain
• If it takes longer to scan greater imagined
  distance (timedistance/speed)
• i.e. simulation of real space

18
PYLYSHYNS RESPONSE TO KOSSLYNcont.

• Other differences in how visual and mental images
  are accessed
• Signature properties
• Manipulations of mental images not accurate
• Cortical images fade more quickly
• Emmerts Law
• Two forms of images not connected to the motor
  system in the same way (reaching for an object)
• Retinal images are pre-interpretation mental
  images are the interpretation

19
PYLYSHYNS RESPONSE TO KOSSLYNcont.

• Cognitive maps are developed by being in the
  environment, receiving cues, interpreting route
  cues as opposed to visual, pictorial cues on a
  map
• Use propositional knowledge to navigate, as it is
  needed
• Can be pictorial, but isnt by nature
• Blind people are able to form cognitive maps
  (Landau, 1986)
• The experience of moving through an environment
  is enough to form a cognitive map, without
  actually navigating the environment (Millar, 1994)

20
PYLYSHYNS RESPONSE TO KOSSLYNcont.

• When one is driving down the M4, one does not
  necessarily visualize a map of the M4 going
  East-West and taking exit 18 south to bath, one
  simply exits at exit 18 and drives (subconscious
  behaviour)
• One accesses ones cognitive map of the region
  without stimulating the visual cortex (however it
  is (hopefully) stimulated by the road ahead but
  not terms of the cognitive map)

21
GLOBAL POSITIONING SYSTEMS

• GPSs are a network of satellites and computers
  that can locate, track and monitor any mobile
  object/individual on earth
• GPS in cars provide the user with a pictorial map
  and auditory directions
• GPS is normally a navigation tool
• However - if we regard GPS as simply an aid for
  cognitive map development

22
IMPLICATIONS FOR THE DESIGN OF GLOBAL POSITIONING
SYSTEMS (GPS)

• if Kosslyn is right
• GPS technology should display visual
  information.
• However, a visual display may turn out to be more
  of a hindrance than a help. Kosslyn (1985) found
  evidence that visual information similar to that
  of a mental representation can actually interfere
  with the cognitive map.

23
IMPLICATIONS FOR THE DESIGN OF GLOBAL POSITIONING
SYSTEMS (GPS)

• if Pylyshyn is right
• GPS technology should provide auditory
  information

24
GPS TECHNOLOGY cond.

• London cabbies
• Accuracy/reliability of GPS??
• Knowledge of road closures, etc.
• GPS instead of training or to aid training?
• Navigation vs. cognitive maps

• Pictorial vs Propositional debate has not been
  resolved yet
• Although taxi drivers may not find GPS useful on
  the job, there is much agreement that it can be
  useful to develop cognitive maps in training.
• Cognitive maps prove to be pictorial or
  propositional in nature (or a combination of
  both). This will have implications as how best to
  develop future technology.

25
REFERENCES

• Jacobson, D., Lippa, Y., Golledge, R.G., Kitchin,
  R., Blades, M. (2001). Rapid development of
  cognitive maps in people with visual impairments
  when exploring novel geographic spaces. Bulletin
  of people-environment studies, 1-8.
• Klein, I., Dubois, J., Mangin, J-F., Kherif, F.,
  Flandin, G., Poline, J-B., Denis, M., Kosslyn, S.
  M., Le Bihan, D. (2004). Retinotopic organization
  of visual mental images as revealed by functional
  magnetic resonance imaging. Cognitive Brain
  Research, 22, 26-31.
• Kosslyn, S. M. (1983). Ghosts in the Minds
  Machine. Creating and using images in the brain.
  George J. McLeod Limited, Toronto.
• Kosslyn, S. M., Ball, T.M. and Reiser, B. J.
  (1978). Visual images preserve metric spatial
  information Evidence from studies of image
  scanning. Journal of Experimental Psychology
  Human Perception and Performance, 4, 47-60.
• Kosslyn, S. M., Pascual-Leone, A., Felician, O.,
  Camposano, S., Keenan, J. P., Thompson, W. L.,
  Ganis, G., Sukel, K. E., and Alpert, N. M.
  (1999). The role of area 17 in visual imagery
  Convergent evidence from PET and rTMS. Science,
  284, 167-170.
• Kosslyn, S. M., Thompson, W. L., and Alpert, N.
  M. (1997). Neural systems shared by visual
  imagery and visual perception A positron
  emission tomography study. NeuroImage, 6,
  320-334.
• Landau, B. (1996). Early map use as an unlearned
  ability. Cognition, 22, 201-223.

26
REFERENCES cont.

• Lynch, K. (1960). The image of the city.
  Cambridge, MA M.I.T. Press.
• Millar, S. (1994). Understanding and representing
  space theory and evidence from studies with blind
  and sighted children. New York Oxford University
  Press.
• Pylyshyn, Z. (2000). Is the "imagery debate"
  over? If so, what was it about? In E. Dupoux
  (ed.). Cognition a critical look. Advances,
  questions and controversies in honor of J.
  Mehler. Cambridge, MA, MIT Press. Available
  online at http//ruccs.rutgers.edu/ftp/pub/papers
  /pylyshyn-mehler.pdf
• Pylyshyn, Z. (2003). Return of the mental image
  are there really pictures in the brain? Trends in
  Cognitive Sciences, 7(3), 113-118.
• Pylyshyn, Z.W. (2002). Mental Imagery In search
  of a theory. Behavioral Brain Science, 25,
  157-238.
• Slotnick, S. D., Thompson, W. L., and Kosslyn, S.
  M. (2005). Visual Mental Imagery Induces
  Retinotopically Organized Activation of Early
  Visual Areas. Cerebral Cortex, 15, 1570-1583.
• Tolman, E.C. (1932). Purposive behavior in
  animals and men. Berkeley, CA University of
  California Press.