Systematic sentence comprehension in a worldembedded connectionist model

1
Systematic sentence comprehension in a
world-embedded connectionist model

• Stefan Frank
• S.L.Frank_at_uva.nl
• Institute for Logic, Language and Computation
• Pim Haselager Iris van Rooij
• Nijmegen Institute for Cognition and Information

2
Systematicity in language

• If you know some sentences, you know many
• Somebody who understands
• Charlie plays chess outside
• Charlie plays hide-and-seek inside
• will also understand
• Charlie plays chess inside
• Charlie plays hide-and-seek outside
• Knowing a language is not like have memorized a
  phrase book

How to explain this property of language?
3
Systematicity and mental representationFodor
Pylyshyn (1988)

• Classical theory
• Mental representations are composed of symbols
• If you understand charlie, play, chess, inside,
  outside play(x,y), place(x,z), ?you cannot help
  but be systematic and understand play(charlie,ch
  ess) ? place(charlie, inside), play(charlie,chess
  ) ? place(charlie, outside), ...
• Classical theory explains systematicity
• Connectionism
• Representations in neural networks are (usually)
  not symbolic and compositional
• So Connectionism cannot explain systematicity
• At best, Connectionism allows for systematicity

4
Connectionist systematicityour explanation

• Neural networks adapt to the training input they
  receive from the environment.
• There is systematicity in the environment
  structure in
• language
• the mapping from language to meaning
• the world
• Systematicity can develop in a neural network
  that is embedded in a systematic world.

We present a connectionist model of sentence
comprehension that explains systematicity
5
Our approach

• Design a microworld.
• Develop non-symbolic, non-compositional
  representations of microworld events.
• Design a microlanguage to describe microworld
  events in.
• Train a neural network to transform microlanguage
  sentences into representation of the described
  events.
• Show that the network is systematic.

6
The microworldconcepts and events

• 22 Concepts, e.g.,
• people charlie, heidi, sophia
• games chess, hideseek, soccer
• toys puzzle, doll, ball
• places bathroom, bedroom, street, playground
• predicates play, place, win, lose
• 44 basic events, e.g., play(charlie, chess),
  win(sophia), place(heidi, bedroom)
• States of the world are (boolean combinations of)
  basic events, e.g.,
• play(sophia, hideseek) ? place(sophia,
  playground)
• sophia plays hide-and-seek in the playground
• lose(charlie) ? lose(heidi) ? lose(sophia)
• someone loses

7
The microworldconstraints and regularities

• There are all kinds of interdependencies among
  events
• Implications, e.g.
• win(p) ? play(p, g)
• place(p, x1) ? ?place(p, x2) (x1 ? x2)
• play(p1,puzzle) ? place(p1,bedroom) ?
  ?play(p2,puzzle) (p1 ? p2)
• Correlations, e.g.
• place(sophia, x) and place(heidi,
  x)place(sophia, x) and ?place(charlie,
  x)play(charlie,soccer) and lose(charlie)play(cha
  rlie,chess) and win(chess)

Within the model, there are no concepts,
predicate-argument structures, or variables. The
smallest meaningful unit in the model is the
basic event. The models representations are
non-compositional.
8
Symbolic vs. analogical representation

• A representation is symbolic if there is an
  arbitrary relation between its form and its
  meaning. (Peirce, 1903)
• The models representations are extracted from
  25,000 observations of microworld situations.
  Any situation is represented by a situation
  vector.
• These representations are analogical
  dependencies among vectors mirror dependencies
  among events.
• Belief values estimates of conditional
  probabilitiesPr(pq) and Pr(qp) follow from
  comparing the situation vectors representing p
  and q.

9
Explicit vs. direct inference

• Suppose p ? q in the world (e.g., sunshine ?
  blue-sky)
• Using symbolic representations, an explicit
  inference process is needed to get from p to q
• An analogical representation of p can also
  represent q, allowing for direct inference of q
  from p
• Situation vectors result in direct inference
  represent play(sophia, soccer) and look at belief
  values for
• play(sophia, ball) .99
• play(sophia, puzzle) 0

shine(sun) shine(sun) ? blue(sky) ? blue(sky)
blue sky!
10
The microlanguage

• 40 Words, e.g.,
• Names charlie, sophia, heidi
• (Pro)nouns girl, boy, someone, street, soccer,
  football, ...
• Verbs plays, loses, ...
• Adverbs inside, outside, ...
• Prepositions

11
The microlanguage

• 40 Words
• 13 556 Sentences, e.g.,
• girl plays chess
• ball is played with by charlie
• heidi loses to sophia at hide-and-seek
• someone wins

12
The microlanguage

• 40 Words
• 13 556 Sentences, e.g.,
• girl plays chess
• play(heidi, chess) ? play(sophia, chess)
• ball is played with by charlie
• play(charlie, ball)
• heidi loses to sophia at hide-and-seek
• win(sophia) ? lose(heidi) ? play(heidi,hideseek)
  
• someone wins
• win(charlie) ? win(heidi) ? win(sophia)
• Semantics each sentence describes one microworld
  situation, which can be represented by a
  situation vector

13
The sentence comprehension model

• Simple Recurrent Network (Elman, 1990)
• input microlanguage sentences (one word at a
  time)
• output situation vectors
• Comprehension score for event p
• the increase in belief value of p resulting from
  processing a sentence
• between -1 and 1
• Sentence states that p
• Score for p should be positive
• Score of inconsistent events shouldbe negative

output (150 units)situation vectors
hidden (120 units)word sequences
input (40 units)words
14
When is a neural network systematic?Hadley (1994)

• Systematicity
• knowing x ? knowing y
• Neural network generalization
• training on input x ? ability to process y

Neural networks are systematic to the extent that
they can handle new (i.e., untrained) input
15
Experiment

• The network was not trained on (among others)
• Any sentence containingheidi beats/loses to
  charliecharlie beats/loses to sophiasophia
  beats/loses to heidi
• Any sentence stating thatplay(p, chess) ?
  place(p, bedroom)play(p, hideseek) ? place(p,
  playground)
• Any sentences stating thatplay(charlie,
  ball) play(heidi, puzzle) play(sophia, doll)

The network displays systematicity if it can
simulate comprehension of these three types of
sentences.
16
Results
training sentences
matched test sentences
17
Results
training sentences
matched test sentences
18
Results
training sentences
matched test sentences
3. Example charlie plays with ball event compr.
score play(charlie,ball) .49 place(charlie,doll)
-.20 play(charlie,puzzle) -.41
19
Conclusions

• The model simulates comprehension of (many) novel
  sentences.
• No implementation of a symbol system.
• Findings are robust, do not depend on
  sophisticated training regime, architecture, or
  parameter setting.
• Neural networks are not inherently systematic, so
  systematicity must have originated elsewhere.
• During training, the network adapts to the
  structure of the environment and, as a result,
  becomes systematic.

Systematicity does not need to be inherent to the
cognitive system.It can originate externally.