Language evolution and robotics: from holistic to compositional language

1
Language evolution and robotics from holistic to
compositional language

• Paul Vogt
• Language Evolution and Computation unit
• University of Edinburgh, UK
• ILK / Computational Linguistics
• Tilburg University, The Netherlands
• http//www.ling.ed.ac.uk/paulv
• paulv_at_ling.ed.ac.uk

2
Symbol grounding problem
Physical symbol system hypothesis PSS are both
sufficient and necessary condition for
intelligence (Newel Simon 1976).
The symbols of a PSS only meaningful to an
external observer.
How do seemingly meaningless symbols become
meaningful ? (Harnad 1990)
From Pfeifer Scheier 1999
Symbols must be grounded in agent-environment
interaction.
3
Embodied cognition
Physical grounding hypothesis Intelligence
should be grounded in the interaction between
a physical agent and its environment (Brooks
1990).
Symbolic representations no longer necessary.
Intelligent behaviour can be established by
parallel operating sensori- motor couplings.
4
Need for symbols

• Human language is a symbolic communication
  system, i.e., a communication system that has an
  arbitrary relation between form and sense.
• Many other cognitive phenomena (such as thinking)
  appear to be symbolic.

5
Semiotic symbols (C.S. Peirce 1935-1958)
Form
EEND
Meaning
Referent
6
Semiotic symbols(C.S. Peirce 1935-1958)

• Meaning arises from the interaction of the form
  with the referent.
• Semiotic symbols are
• situated they are constructed from
  agent-environment interaction.
• embodied they are based on (a history of) bodily
  experiences.
• meaningful from within.

7
Physical symbol grounding

• Symbol grounding problem no longer a fundamental
  problem.
• Symbol grounding problem shifts toward technical
  problem physical symbol grounding problem
  Constructing the semiotic triangle

8
Language evolution and robotics

• We have to have a
• model of evolution with a multi-agent system.
• population of agents/robots situated in the
  physical world (possibly experimental or even
  simulated environment).
• individual agents who construct semiotic symbols
  about relevant events or objects in the world
  i.e. they have to solve the physical symbol
  grounding problem.
• sharing mechanism such as cultural interaction
  and individual adaptation.

9
Short history

• Evolution of communication
• Communication channels (Quinn 2001)
• Lexicon grounding
• Mobile robots (Steels Vogt 1997 Vogt 2000)
• Talking Heads (Belpaeme et al. 1998 Steels et
  al. 2002)
• Robot arm (Marocco, Nolfi Cangelosi 2003)
• Grammatical structures
• Case-grammar (Steels 2003)
• Compositionality (Vogt 2003)

10
Objectives

• Not
• Mimicking human language evolution
• Human-like languages
• Human-like concepts

• But
• Trying to make a point
• New theories
• Verifying theories
• Studying aspects of language evolution.
• Modeling these (i.e. simplifying a lot)

11
Overview

• Physical symbol grounding
• Lexicon grounding
• Language game model
• Results
• Towards compositional structures
• Iterated learning
• Results
• Constructing symbols meaningfully
• Conclusions

12
Language as a complex dynamical system

• Cultural evolution (no innate Universal Grammar,
  universal tendencies emerge through cultural
  transmission)
• Individual adaptation (learning mechanisms are
  likely innate)
• Self-organisation (a global structure language
  emerges from local interactions)
• Co-evolution of language and meaning (emergence
  of semantic structures give rise to the emergence
  of syntactic structures v.v.)

13
Talking Heads (Steels et al. 2002)

• Setup with two cameras on a tripod.
• Each camera resembles a Talking Head.
• PowerMac for processing
• Environment geometrical figures on white-board.
• Experiment Language evolution on the Internet
  (largely uncontrolled, because interaction with
  human users)

14
THSim - Talking Heads simulation tool
http//www.ling.ed.ac.uk/paulv/thsim.html
15
Language games
16
Topic selection
17
Feature extraction
(Red, Green, Blue, Shape)
18
Categorisation
1-nearest neighbourhood search
19
Discrimination game
(1,1,0,1)
(1,1,0,1)
20
LEXICON
yellowbike
(1,1,0,1)
yellowsquare
(0,0,1,1)
bluesquare
bluecircle
redsquare
21
Production
LEXICON
yellowbike
(1,1,0,1)
(1,1,0,1)
yellowsquare
yellowsquare
yellowsquare
(0,0,1,1)
bluesquare
bluecircle
redsquare
22
Interpretation
LEXICON
Blue square
(1,0,0,1)
(0,0,1,1)
yellowsquare
(0,0,1,0)
(1,1,0,.8)
(1,1,0,1)
23
Invention
LEXICON
yellowbike
(1,1,0,1)
yellowsquare
(0,0,1,1)
bluesquare
bluecircle
(1,1,0,1)
24
Adoption
LEXICON
Blue square
(1,0,0,1)
(0,0,1,1)
yellowsquare
(0,0,1,0)
(1,1,0,1)
redsquare
(1,0,0,1)
25
Adaptation
LEXICON
Blue square
(1,0,0,1)
(0,0,1,1)
yellowsquare
(1,1,0,1)
(1,1,0,.8)
(1,1,0,0)
26
Adaptation
LEXICON
Blue square
(1,0,0,1)
(0,0,1,1)
yellowsquare
(1,1,0,1)
(1,1,0,.8)
(1,1,0,0)
27
GAVAGAI
Observational game joint attention / Hebbian
learning
Guessing game corrective feedback
/ reinforcement learning
Selfish game cross-situational / statistical
learning
28
Robotic experiments
29
Results

• Guessing games 75-80
• Within limits almost perfect
• Observational games 75-80
• Within limits almost perfect
• Selfish games 23
• a priori chance

30
Results

• Guessing games 90-100
• Observational games 100
• Selfish games 75

31
Conclusions

• Symbol grounding as interaction between agent and
  its environment
• Lexicon grounding is possible in relatively small
  environment, provided there is some form of mind
  reading.

32
Modelling the transition from holistic to
compositional languages

• Holistic languages languages WITH NO structural
  relationship between parts of an expression and
  parts of its meaning
• S ? wateve / redsquare
• Compositional languages languages WITH a
  structural relation between parts of an
  expression and parts of its meaning
• S ? wat / red eve / square

33
Evolving compositional syntax through Iterated
Learning (Brighton, Kirby)
M
M
L
H1
H2
Adult
Learner
34
Evolving compositional syntax through Iterated
Learning (Brighton, Kirby)
M
H2
Adult
Learner
35
Iterated learning and grounding
M
M
M
L
L
L
H1
H2
H3

A1
A2
A3
36
Co-development syntax semantics
Compositional semantic structures are shaped from
regularities in agents interactions with their
ecological niche, though constrained by syntactic
structures. Compositional syntactic structures
are shaped from regularities in linguistic
expressions, though constrained by semantic
structures.
37
Basic idea
Shape
Red
X-axis
Green
Y-axis
Blue
38
Finding conceptual spaces
(1,1,0,0.57)
yellowcircle
yellowsquare
(1,1,0,1)
(0,0,1,1)
bluesquare
39
Simulations

• Population each iteration 1 adult, 1 learner
• At start of agents lifetimes
• no meanings, no words, no grammar
• 10 trials 5000 iterations of 150 language games
• Each game construction of context containing 8
  randomly generated objects from set of 120
  objects (12 colours x 10 shapes)
• Maximum of 2 constituents per sentence

40
Results sim / comp
41
Results - Grammar

• S ? A / rgb B / s and S ? B / s A / rgb most
  dominant rules
• However, grammar is instable. Over different
  iterations there occur many changes
• in word-order
• in (dominant) segmentation of conceptual spaces
• in ways of splitting words

42
Conclusions

• Emergence of compositional structures by
  exploiting regularities in both linguistic
  utterances and interactions with the ecological
  niche.
• No transition from holistic to compositional
  structures observed between generations, but
• compositional structures emerged within
  iterations
• S ? A / rgb B / s and S ? B / s A / rgb most
  dominant rules, reflecting most dominant
  regularity of ecological niche

43
Viability experiments
(Steels 1994)
44
Exploiting control mechanisms
Marilyn Monroes (Key West, Florida, 1995) by
Peter Krogh (Nat. Geographic)
Development of functions such as using language
would be rather simple once the essence of being
and reacting are available (Brooks, 1990,
Elephants don't play chess)
45
Extended Braitenberg vehicle
Phototaxis charging station
Phototaxis competitors
charge
Energy level low ?
Activate PCS
Activate charge
46
Extended Braitenberg vehicle
Phototaxis charging station
Phototaxis competitors
compete
Energy level high ?
Activate PC
Activate compete
47
Simulations

• Environment 4 visually distinct charging
  stations, two agents
• Agents must visit charging stations
  simultaneously to refill energy supplies.
• 10 trials, 5000 guessing games, no iterated
  learning, no learning of viability task

48
Results
49
Representation and meaning

• Meaning representation
• ltAction-schemaapproach-target,
• Target-colourredgt
• Utterance
• wabako
• Meaning
• Speaker I will go to the red target
• Hearer I think the speaker goes to the red
  target, and so will I

50
Conclusions

• Grounding (basic) actions can be achieved by
  associating signals with activations of reactive
  sensorimotor couplings.
• Meaning emerges as higher-order interpretation
  of some representation through the interaction of
  agent with its environment

51
Language evolution and robotics

• We are capable of
• Lexicon grounding
• Evolving extremely simple grammars
• Evolving cooperative behaviour using evolved
  language
• Open questions
• Relation to human behaviour?
• Scalability (lexicons, grammars, coop.
  behaviour)?
• Emergence of meaningful/intentional behaviour?
• Emergence of more complex grammatical
  constructions?

52
(No Transcript)