Lecture Overview

1
Lecture Overview

• Regier System Limitations
• Image Schemas Recap
• Force Dynamic Schemas Recap
• Sensory-Motor Schemas
• Evidence in Primates
• Evidence in Humans
• Do motor schemas play a role in language?
• A Computational Model of Motor Schemas
• Learning Hand Action terms (Bailey)
• Cultural Schemas and frames

2
Limitations

• Scale
• Uniqueness/Plausibility
• Grammar
• Abstract Concepts
• Inference
• Representation

3
Force Dynamics, modals and causatives

• A gust of wind made the pages of my book turn.
• The appearance of the headmaster made the pupils
  calm down.
• The breaking of the dam let the water flow from
  the storage lake.
• The abating of the wind let the sailboat slow
  down.

4
Talmys force dynamic schemas

• Two entities in a forceful interaction
• Agonist (foregrounded)
• Antagonist (has an effect on the Agonist)
• Each entity exerts a force on the other
• An entity has an tendency
• towards rest (inaction)
• toward motion
• Opposed forces have different relative strength
  
• Depending on the relative strength the result
  of the interaction could make the Agonist move
  or be at rest.

5
Schematic Representation(Talmy)
6
FD Patterns

• A gust of wind made the pages of my book turn.
• The appearance of the headmaster made the pupils
  calm down.
• The breaking of the dam let the water flow from
  the storage lake.
• The abating of the wind let the sailboat slow
  down.

7
Semantic field Force-dynamics representation
Physical The ball kept rolling along the green
Physical/psychological  John can't go out of the house
Intra-psychological He refrained from closing the door
Intra-psychological (lexicalized) She's civil to him 
Socio-psychological  She gets to go to the park
8
An Experiment with Force Dynamics
9
Language Specific Bias
10
Closed Class vs. Open Class terms

• Image Schematic and Force Dynamic Patterns are
  expressed by closed class terms in language
• Prepositions (in, on, into, out)
• Modals and causatives (make, let, might, prevent)
• How about open class terms?
• Verbs and Event descriptions
• Motor Schemas - Embodied
• Is there evidence for motor schemas and if so are
  they used in language?
• Frames Composed from Image and motor schemas
  -Cultural

11
Coordination

• PATTERN GENERATORS, separate neural networks that
  control each limb, can interact in different ways
  to produce various gaits.
• In ambling (top) the animal must move the fore
  and hind leg of one flank in parallel.
• Trotting (middle) requires movement of diagonal
  limbs (front right and back left, or front left
  and back right) in unison.
• Galloping (bottom) involves the forelegs, and
  then the hind legs, acting together

12
Sensory-Motor Schemas

• A sensory (perceptual) schema determines whether
  a given situation is present in the environment.
  
• Object Detection
• Spatial relation recognition
• Execution of current plans is made up of motor
  schemas which are akin to control systems but
  distinguished by the fact that they can be
  combined to form coordinated control programs
• Sensory and Motor Schemas are closely coupled
  circuits sensory-motor schemas.

13
The neural theory

• Human concepts are embodied. Many concepts make
  direct use of the sensory-motor capacities of our
  body-brain system.
• Many of these capacities are also present in
  non-human primates.
• Let us look at concepts that make use of our
  sensory-motor capacities, ex. Grasp.

14
Area F5
General Purpose Neurons General Grasping General
Holding General Manipulating
15
General Purpose Neurons in Area F5
A Grasping with the mouth B Grasping with the
cl. hand C Grasping with the ipsil. hand
(Rizzolatti et al. 1988)
16
General Purpose Neurons Achieve Partial
Universality Their firing correlates with a
goal-oriented action of a general type,
regardless of effector or manner.
17
Area F5c
Convexity region of F5 Mirror neurons
18
F5c-PF
Rizzolatti et al. 1998
19
F5 Mirror Neurons
tools
Gallese and Goldman, TICS 1998
20
Strictly congruent mirror neurons (30)
Observed Action
Executed Action
Executed Action
(Rizzolatti et al. Cog Brain Res 1996)
21
Category Loosening in Mirror Neurons (60)
(Gallese et al. Brain 1996)
A C is Observe (Execute) Precision Grip
(Prototype) B D is Observe (Execute) Whole
Hand Pre-hension
22
The F5c-PF circuit Links premotor area F5c and
parietal area PF (or 7b). Contains mirror
neurons. Mirror neurons discharge when Subject
(a monkey) performs various types of goal-related
hand actions and when Subject observes another
individual performing similar kinds of actions
23
Phases Area F5 contains clusters of neurons that
control distinct phases of grasping opening
fingers, closing fingers. Jeannerod, et al.,
1995 Rizzolatti, et al., 2001.
24
Mirror Neurons Achieve Partial
Universality, since they code an action
regardless of agent, patient, modality
(action/observation/hearing), manner,
location. Partial Role Structure, since they
code an agent role and a purpose role. The
Agent Role In acting, the Subject is an agent
of that action. In observing, the Subject
identifies the agent of the action as having the
same role as he has when he is acting namely,
the agent role. The Purpose Role Mirror neurons
fire only for purposeful actions.
25
Mirror Neurons Achieve Category tightening and
loosening
26
The F4-VIP circuit
27
The F4-VIP Circuit Links premotor area F4 and
parietal area VIP. Transforms the spatial
position of objects in peri-personal space into
motor programs for interacting with those
objects. Examples Reaching for the objects, or
moving away from them with various parts of your
body such as the arm or head.
28
Area F4
Arm reaching Head turning
29
Somato-Centered Bimodal RFs in area F4
(Fogassi et al. 1996)
30
Somato-Centered Bimodal RFs in area VIP
(Colby and Goldberg 1999)
31
Somato-Centered Bimodal RFs in area F4
(Fogassi et al. J Neurophysiol 1996)
32
AIP and F5 (Grasping) in Monkey
AIP - grasp affordances in parietal cortex Hideo
Sakata
F5 - grasp commands in premotor cortex Giacomo
Rizzolatti
33
Size Specificity in a Single AIP Cell

• This cell is selective toward small objects,
  somewhat independent of object type ( Hideo
  Sakata)
• Note Some cells show size specificity others
  do not.

34
Summary of Fronto-Parietal Circuits

• Motor-Premotor/Parietal Circuits
• PMv (F5ab) AIP Circuit
• grasp neurons fire in relation to movements
  of hand prehension necessary to grasp object
• F4 (PMC) (behind arcuate) VIP Circuit
• transforming peri-personal space coordinates so
  can move toward objects
• PMv (F5c) PF Circuit F5c
• different mirror circuits for grasping, placing
  or manipulating object
• Together suggest cognitive representation of the
  grasp, active in action imitation and action
  recognition

35

• MULTI-MODAL INTEGRATION
• The premotor and parietal areas, rather than
  having
• separate and independent functions, are neurally
  integrated
• not only to control action, but also to serve the
  function of
• constructing an integrated representation of
• Actions, together with
• objects acted on, and
• locations toward which actions are directed.
• In these circuits sensory inputs are transformed
  in order to
• accomplish not only motor but also cognitive
  tasks, such as
• space perception and action understanding.

36
Modeling Motor Schemas

• Relevant requirements (Stromberg, Latash, Kandel,
  Arbib, Jeannerod, Rizzolatti)
• Should model coordinated, distributed,
  parameterized control programs required for motor
  action and perception.
• Should be an active structure.
• Should be able to model concurrent actions and
  interrupts.
• Should model hierarchical control (higher level
  motor centers to muscle extensor/flexors.
• Computational model called x-schemas
  (http//www.icsi.berkeley.edu/NTL)