Connectionist Models: Basics

1
(No Transcript)
2
Dealing with Implicit Negatives

• Explicit positive for above
• Implicit negatives for below, left, right, etc
• in Regier
• E ½ ?i,p (( ti,p oi,p) ßi,p )2,
• where i is the node, p is the pattern,
• ßi,p 1 if explicit positive,
• ßi,p lt 1 if implicit negative

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
Learning System
dynamic relations (e.g. into)
structured connectionistnetwork (based on
visual system)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
Topological Relations

• Separation
• Contact
• Coincidence
• Overlap
• Inclusion
• Encircle/surround

18
Issue 2 Shift Invariance

• Backprop cannot handle shift invariance (it
  cannot generalize from 0011, 0110 to 1100)
• But the cup is on the table whether you see it
  right in the center or from the corner of your
  eyes (i.e. in different areas of the retina map)
• What structure can we utilize to make the input
  shift-invariant?

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Limitations

• Scale
• Uniqueness/Plausibility
• Grammar
• Abstract Concepts
• Inference
• Representation

24
Demo of the Regier System

• on the English above

25
Language and Thought

• We know thought (our cognitive processes)
  constrains the way we learn and use language
• Does language also influence thought?
• Benjamin Whorf argues yes
• Psycholinguistics experiments have shown that
  linguistics categories influence thinking even in
  non-linguistics task

Language
Thought
cognitive processes
26
5 levels of Neural Theory of Language
Spatial Relation
Motor Control
Metaphor
Grammar
Cognition and Language
Computation
Structured Connectionism
abstraction
Neural Net
SHRUTI
Computational Neurobiology
Triangle Nodes
Biology
Neural Development
Midterm
Quiz
Finals
27
Language, Learning and Neural Modelingwww.icsi.be
rkeley.edu/AI

• Scientific Goal
• Understand how people learn and use language
• Practical Goal
• Deploy systems that analyze and produce
  language
• Approach
• Build models that perform cognitive tasks,
  respecting
• all experimental and experiential
  constraints
• Embodied linguistic theories with advanced
• biologically-based computational methods

28
Constrained Best Fit in Nature
inanimate animate
physics lowest energy state
chemistry molecular minima
biology fitness, MEU Neuroeconomics
vision threats, friends
language errors, NTL
29
Simulation-based language understanding
Utterance
Harry walked to the cafe.
Constructions
Analysis Process
General Knowledge
Simulation Specification
Schema Trajector Goal walk Harry cafe
Belief State
Simulation
30
Simulation Semantics

• BASIC ASSUMPTION SAME REPRESENTATION FOR
  PLANNING AND SIMULATIVE INFERENCE
• Evidence for common mechanisms for recognition
  and action (mirror neurons) in the F5 area
  (Rizzolatti et al (1996), Gallese 96, Boccino
  2002) and from motor imagery (Jeannerod 1996)
• IMPLEMENTATION
• x-schemas affect each other by enabling,
  disabling or modifying execution trajectories.
  Whenever the CONTROLLER schema makes a transition
  it may set, get, or modify state leading to
  triggering or modification of other x-schemas.
  State is completely distributed (a graph marking)
  over the network.
• RESULT INTERPRETATION IS IMAGINATIVE SIMULATION!

31
Psycholinguistic evidence

• Embodied language impairs action/perception
• Sentences with visual components to their meaning
  can interfere with performance of visual tasks
  (Richardson et al. 2003)
• Sentences describing motion can interfere with
  performance of incompatible motor actions
  (Glenberg and Kashak 2002)
• Sentences describing incompatible visual imagery
  impedes decision task (Zwaan et al. 2002)
• Simulation effects from fictive motion sentences
• Fictive motion sentences describing paths that
  require longer time, span a greater distance, or
  involve more obstacles impede decision task
  (Matlock 2000, Matlock et al. 2003)

32
Neural evidence Mirror neurons

• Gallese et al. (1996) found mirror neurons in
  the monkey motor cortex, activated when
• an action was carried out
• the same action (or a similar one) was seen.
• Mirror neuron circuits found in humans (Porro et
  al. 1996)
• Mirror neurons activated when someone
• imagines an action being carried out (Wheeler et
  al. 2000)
• watches an action being carried out (with or
  without object) (Buccino et al. 2000)

33
Area F5c
Convexity region of F5 Mirror neurons
34
F5 Mirror Neurons
Gallese and Goldman, TICS 1998
35
Category Loosening in Mirror Neurons (60)
Observed A is Precision Grip B is Whole Hand
Prehension Action C precision grip D
Whole Hand Prehension
(Gallese et al. Brain 1996)
36
PF Mirror Neurons

• Neuron responds to execution (grasping) but to
  grasping and releasing in observation.
• Mirror neurons in parietal cortex.
• Difference in left hand and right hand.

(Gallese et al. 2002)
37
A (Full vision) B (Hidden) C (Mimicking) D
(HiddenMimicking)
Umiltà et al. Neuron 2001
38
F5 Audio-Visual Mirror Neurons
Kohler et al. Science (2002)
39
Somatotopy of Action Observation
Foot Action
Hand Action
Mouth Action
Buccino et al. Eur J Neurosci 2001
40
The Mirror System in Humans
BA6
41
The ICSI/Berkeley Neural Theory of Language
Project
ECG
42
(No Transcript)
43
Computing other relations

• The 2/3 node is a useful function that activates
  its outputs (3) if any (2) of its 3 inputs are
  active
• Such a node is also called a triangle node and
  will be useful for lots of representations.

44
Triangle nodes and McCullough-Pitts Neurons?
A
B
C
45
Representing concepts using triangle nodes
triangle nodes when two of the neurons fire, the
third also fires
46
They all rose

• triangle nodes
• when two of the neurons fire, the third also
  fires
• model of spreading activation

47
Basic Ideas behind the model

• Parallel activation streams.
• Top down and bottom up activation combine to
  determine the best matching structure.
• Triangle nodes bind features of objects to values
• Mutual inhibition and competition between
  structures
• Mental connections are active neural connections

48
Can we formalize/model these intuitions

• What is a neurally plausible computational model
  of spreading activation that captures these
  features.
• What does semantics mean in neurally embodied
  terms
• What are the neural substrates of concepts that
  underlie verbs, nouns, spatial predicates?

49
Spreading activation and feature structures

• Parallel activation streams.
• Top down and bottom up activation combine to
  determine the best matching structure.
• Triangle nodes bind features of objects to values
• Mutual inhibition and competition between
  structures
• Mental connections are active neural connections

50
Feature Structures in Four Domains
Barrett Ham Container Push
deptCS Color pink Inside region Schema slide
sid001 Taste salty Outside region Posture palm
empGSI Bdy. curve Dir. away

Chang Pea Purchase Stroll
deptLing Color green Buyer person Schema walk
sid002 Taste sweet Seller person Speed slow
empGra Cost money Dir. ANY
Goods thing