Sensory modalities

1
Sensory modalities

• Different approaches

2
ORegan, J. K., Noe, A., A sensorimotor account
of vision and visual consciousness

• Behavioral and Brain sciences (2001), 245

3
Unsignificant internal architecture

• There is no internal representations or
  architecture to explain the conscious quality of
  different sensations
• From the point of view of the brain there is
  nothing that in itself differentiates nervous
  influx coming from retinal, haptic,
  proprioceptive, auditory, and other senses
• Even if the size, shape, firing patterns, places
  wqwhere neurons are localized in the cortex
  differ, this does not, in itself confer them with
  any particular visual, olfactory, motor or other
  perceptual qualities

4
Sensorimotor contingencies

• What does explain the differences between
  sensations is the structure of the rules
  governing the sensory changes produced by various
  motor actions sensorimotor contingencies
• Within different sensory modalities, sensorimotor
  contingencies differ because they are subject to
  different invariance properties
• Each sensory modality is defined by a particular
  structure of the rules governing sensoru changes
  produced by movement
• Sensory modalities are modes of exploration of
  the world mediated by knowledge of sensorimotor
  contingencies

5
Classes of sensorimotor contingency

• We can destinguish between 2 classes of
  sensorimotor contingency
• Sensorimotor contingencies determined by the
  character of the visual apparatus (the eye is a
  kind of sphere, etc.)
• They are the fundamental aspect of visual
  sensation (because they are independent of any
  interpretation or categorization of objects)
• Sensorimotor contingencies determined by the
  visual attributes of the object, or geometric
  invariants (characteristics of the object that
  doesnt change when the eye moves)
• They are the basis of visual perception

6
Visual sensorimotor contingencies visual percepts

• Movements of the eyes
• When the eyes rotate, the sensory stimulation of
  the retina shifts and distorts in a very
  particular way, determined by the size of the eye
  movement, the shape of the retina, the nature of
  the ocular optics contours shift and the
  curvature of the lines changes, the distribution
  of information on the retin changes (because of
  the difference of distribution of photoreceptors
  in central and perripheral vision)
• When the body moves forwards, the flow pattern on
  the retina is an expanding one when moving
  backwards a contracting one
• When the eyes close during blinks, the
  stimulation changes drastically becoming uniform
• Constraints of visual-type exploration/visual
  attributes of objects
• The retinal image of an object only provides a
  frontal view of the object
• When one moves around the object parts of the
  object appear and disappear
• We can only apprehend an object from a definite
  distance (retinal projection depends on distance)
• Color and brightness of the objects change when
  one moves around the object or lighting
  conditions change

7
Perception of visual shape

• Exemple Patiens born blind and operated
• A patient cited by Helmholtz is surprised that a
  coin should so dramatically change its shape when
  rotated
• A patient of Cheseldon was surprised by the
  change in extent of an object in fonction of the
  distance
•  The visual quality of shape is precisely the
  set of all potential distortions that the shape
  undergoes when it is moved relative to us, or
  when we move relative to it.
• Although it is an infinite set, the brain can
  abstract from this set a series of laws, and it
  is this set of laws which codes shape 

8
Other sensorimotor contingecies auditory percepts

• Auditory sensorimotor contingencies are not
  affected by eye movements or blinks, but they are
  affected in a special way by head movements
• rotations of the head change the temporal
  asynchrony between the ears
• movements of the head in the direction of the
  sound source affect the amplitude but not the
  frequency of the input
• Tactile exploration of an object obeys different
  sensorimotor contingencies
• One doesnt touch an object from a point of view
• The tactile aspect of an object doesnt change
  with lighting conditions

9
Knowledge of sensorimotor laws

• Summary  what differentiate the senses are not
  the neural pathways but the laws obeyed by the
  sensorimotor contingencies associates with these
  senses 
• Sensorimotor contingencies are distinctive for
  each sensory modality
• The laws of sensorimotor contingency are
  determined by the fact that the exploration is
  being made by a particular apparatus and
  exercised on specific characters of the object
• The brain distinguishes between the senses on the
  basis of the different sensorimotor laws a
  percept respond to
• Two conditions must be achieved to perceive in a
  certain modality
• Presence of the conditions for the two kind of
  sensorimotor contingency laws
• Active master of the laws
• Third condition for visual (sensory) awareness
• The master of the laws must be comprised in
  plannings, thougs, linguistic behavior

10
A practical form of knowledge

• The mastery of the rules of sensorimotor
  contingency is not something we possess a
  propositional knowledge about
• We are not able to describe in detail how it is
  distorted a seen surface when we rotate the eyes
• The deviation from the laws that our brain
  normally extract causes an impression that
  something unusual is happening and let us see the
  distortions
• When we put on a new pair of glasses we see
  distortions for a while because the movement of
  the eyes provokes displacements of unusual
  amplitude
• The knowledge of the laws is a predictive or
  anticipatory knowledge  Perceptual experience
  is a mode of activity involving practical
  knowledge about currently possible behaviors and
  associated sensory consequences 
• When one sees or touch a bottle there is a
  knowledge in his nervous system about what he
  would feel or see if he move his hands or eyes
  upwards or downwards the bottle, and it is this
  anticipation that tells him that he is touching a
  bottle and not a pitcher

11
Some phenomena explained within sensorimotor
contingency approach

• Tactile visual sensory substitution
• In experiences with TVSS it is only when the
  observer is allowed to move the videocamera that
  identification becomes possible and that that he
  begins to  see  something as externally
  localized, in the space before him (also if the
  stimulus array is on his back)
• If an experience in one sensory modality derives
  from the  rules that govern action-related
  changes in sensory input  visual experience
  could be obtainable by other sensory channels
  when the brain extracts the same invariants in
  the structure of the senorimotor contingencies
• So subjects with TVSS see and not just  see 
  the fact that the stimulus is provided by the
  skin is irrelevant, provided that the stimulation
  obeyes the required sensorimotor laws (of course
  the laws are not exactly the same as seeing with
  the eyes, as seeing in the dark is not the same
  thing that seein with full light)

12
Multimodal interactions

• Synesthesia and other multimodal substitutions
  and interactions as ventriloquism effect, McGurk
  effect and others
• The experience associated with a modality exists
  only within the context of other senses available
  to the organism senses are not really separated
• Although sensory modalities have their own
  specificities dues to the particularities of
  sensors and of sensorimotor contingencies, when
  there are systematic correlations and common
  sensorimotor contingencies, interactions between
  the senses are to be expected
• Anyway, the mentioned specificities define the
  particular experience associated with each sense

13
Binding problem

• The binding problem is a pseudo problem
• Neuroanatomists believe that the visual system is
  composed of numerous sub-systems or modules which
  are independent and often localized in different
  parts of the brain so they raise the question of
  where and how the different features ultimately
  come together to produce a unified perception of
  reality as we experience they suggest some
  solutions
• The grand-mother cell in which single cells at
  least combine information
• The synchronized oscillation of the separate
  cortical areas which are analysing different
  components of the information
• The binding problem can be analysed under two
  issues
• Temporal unit or coherence
• scientists think that, because we have the
  impression that we see simultaneously all the
  attributes when we identify an object, then the
  information about these attributes must be bound
  together synchronously.
• It is a fallacy physical synchrony is not
  necessary for having a synchronous experience
  because it is not the  internal representation 
  which counts for the coherence of the experience.
   What counts for a temporally coherent
  experience is the fact is a thing we are doing,
  and we are doing it now 
• Conceptual unit or coherence
• The fact that object attributes seem to be part
  of a single object doesnt require them to be
   represented  in a unified way, for example at
  a single location of the brain, or by a single
  process and if they are represented in a
  spatially or temporally way this doesnt explain
  the perceptual coherence.  What explains the
  conceptual unity of experience is the fact that
  experience is a thing we are doing and we are
  doing it with respect to a conceptually unified
  external world 

14
Qualia

• Qualia doesnexist the differences in the
  qualitative character of perceptual experiences
  correspond to differences in the character of the
  relevant sensorimotor contingencies
• The difference between driving a Porche and
  driving a tank is in the different things we do
  driving them, in the different practical
  knowledge about driving them
• The difference between seein and smelling a red
  flower consists in the different patterns of
  sensorimotor contingencies that governs
  perceptual encounte with each
• To experience the redness of an object or to feel
  to drive a Porche is to know what would happen if
  the light changes or if the accelerator is
  pressed down

15
Some problems with multimodality and
intermodality is a sensorimotor account complete?

• The fact that the laws of sensorimotor
  contingency are different from a modality to
  another, doesnt indicate the necessity of a kind
  of binding, even if not an internal binding, but
  a description of the way in which different
  sensorimotor contingencies become an object
  instead of two objects?
• Supposed that we accept there is no need of
  binding, what about the need for coherence could
  a system act in a world that is not coherent?
  Dont we need mechanism to warranty coherence?
  These mechanisms could act on information or on
  sensorimotor contingencies, in order to set an
  accord between contingencies?
• Are there laws that rule  multisensory
  contingency ? Are these rules effective on
  perception? ie regarding the relation between
  vision and touch in prehension complex rules for
  a motor act that define a sort of sensorimotor
  act?
• In tactile visual substitution, ie, how can we
  say that the laws of sensorimotor contingency are
  analogous with eyes or tactile substitution?
  Performances are they better if the device is
  placed on the front of the subject instead of on
  the back (in order to move the camera as he moves
  his head)? Visibly, not