Evolutionary Robotics

1
Evolutionary Robotics

2
(No Transcript)
3
Why Robotics, Artificial Intelligence, or any
Engineering-based approach?

• Psychobiology is in the business of
  reverse-engineering cognitive systems
• Top-down vs. Bottom-up approaches
• Emergentism raises problems for reductionism
• Constructionism
• Encourages specificity
• Epistemological Barriers

4
Braitenbergs Vehicles
5
(No Transcript)
6
Why Evolutionary?

• Functional Decomposition
• Optimization vs. Good-Enough
• Exaptation (Evolution is a miser)
• Natural Selection provides a weakly constraining
  fitness function

7
Architectures

• Representationalism
• Cognitive Systems deal rules operating over
  symbols, specifically propositions, e.g. the
  black dog
• Knowledge-based Systems
• One module contains a warehouse of knowledge,
  stored propositionally
• Another module queries and searches the knowledge
  base for relevant material

8
Architectures

• Production Systems (ACT-r)
• Operate via condition-action rules
• If (predator), then (fear activation)
• If (fear activation), then (run)
• If (run), then (increase heart rate)
• Multiple modules for domain specific tasks
• Working memory buffer
• Perception module
• Action module
• Global behavior emerges from interactions of
  components
• Symbolic systems are very powerful

9
Architectures

• Behavior-based Robotics
• Global behavior of robot emerges through the
  interaction between basic behaviors and the
  environment
• Basis behaviors implemented in separate parts of
  the control system and a coordination mechanism
  determines relative strength of each behavior in
  the particular situation

10
Problems

• Production Systems and Behavior Based Robotics
  are not wrong, just incomplete
• Behaviorism capitalized on stimulus-response
  associations
• Need an architecture sensitive to the structural
  and statistical properties of its inputs
• Need an architecture that has the capacity for
  self-organization (removing the burden of
  explicit design from the experimenter)

11
Problems

• Often, behavior is built into the system rather
  than emerging from the system, thus limiting its
  explanatory value
• Working memory buffer is constrained to only hold
  7-2 items
• What is the propositional representation of a
  picture?
• Functional Decomposition

12
Neural Networks

• McCulloch and Pitts- 1943
• We know that the neuron is the fundamental
  computational unit in the brain
• We know that cognitive abilities arise from
  collections of neurons acting together (lesions)
• How do collections of interconnected neurons
  produce coherent behavior?
• Computer simulations of neurons and collections
  of neurons may tell us

13
(No Transcript)
14
(No Transcript)
15
Logical Operators
AND Sally went to the beach and drank a coke.

T2
1
1
OR Sally will either live or die.
T1
1
1
16
Properties of Neural Networks

• Associative
• Networks are inherently associative. Activity in
  one node spreads to neighboring nodes, activating
  their respective representations.
• Because of this associativity, and the properties
  of distributed representations, similar
  representations cluster.
• Example Hypercolumns Retinotopy

17

• Pattern Completion
• Networks can complete known patterns on the basis
  of partial information. If several units from a
  particular well-known pattern are activated, but
  a few are not, the activation reverberates
  through the network, causing the missing
  information to be completed in the manner most
  consistent with the stimulus information given.
• This property also allows them to compensate for
  noisy input

18

• . Graceful Degredation
• If you destroy a piece of the CPU of a computer,
  it will crash. Or, if you delete a few lines of
  code from a program, it will also crash.
• Brains are not like this.
• Neurons die all the time, dont radically disrupt
  functioning.
• Lesions cause focal deficits
• Because any given neuron or piece of cortex is
  only one of many players in a representation of
  cognitive function, damage has limited effects.
  Deficits increase with increased damage, but its
  not catastrophic

19
Learning

• 1949 Donald Hebb
• Networks can learn by altering the synaptic
  strength, or the weights of connections between
  units.
• However, prior to the 80s, connection weights had
  to be hand-set by trial-and-error to get the
  network to perform a task.
• The trick is to devise a rule that specifies how
  to adjust the weights as a function of past
  performance so that improvement takes place.

20
Supervised Learning

• A teacher provides feedback to a network on its
  performance.
• The teacher may be a set of nodes with the
  correct output for a given problem. A network
  then tries to reach that output given a set of
  inputs.
• An error signal is computed which calculates the
  difference between the actual output (teacher)
  and the arrived at solution (learner).
• Aspects of motor system do this actual vs.
  intended output

21
Backpropogation

• An algorithm which assigns blame to nodes for
  the amount of error.
• Determines which connection weights contributed
  most to the error and sdjusts them.
• Process iterated until no or minimal error
  remains.

22
Reinforcement Learning

• Network (learner) is not told the correct output,
  only whether the arrived at solution is good or
  bad.
• Example The dopaminergic, reward system.

23
Unsupervised Learning

• Self-Organization
• No teacher or reinforcement.
• The local, causal dynamics of the network shape
  its behavior.
• Hebbian Learning

24
The Power of Learning in NNs

• In production systems and engineering, a problem
  is solved in advance, then implemented.
• Neural networks only receive inputs and desired
  outputs, and finds solutions on its own.
• Often, the derived solution is very unintuitive.

25
Emergenesis in a Neural Network
Invalidly-Cued Trial
Validly-Cued Trial
cue
x
x
target
26
Alert
Interrupt
Localize
RT
Disengage
Move
neutral
valid
invalid
Engage
Inhibit
27
Response Unit
Object Unit
Spatial Units
Input Unit
28
Khepera Robot
29
(No Transcript)
30
Genetic Algorithms

• Operates on a population of artificial
  chromosomes by selectively reproducing
  chromosomes of individuals with higher
  performance and applying random changes
• Applied for many generations until fitness
  function stops increasing, or a satisfactory
  individual is found

31
(No Transcript)
32
Artificial Chromosome

• An artificial chromosome is a string that encodes
  the characteristics of an individual
• String may encode the value of a variable of a
  function that must be optimized, may encode
  connection weights of a neural network, or
  network architecture with learning rules for
  network development, etc.
• Most of the subsequent experiments encode
  synaptic weights, so that in essence, multiple
  networks are explored
• How and what to encode in the chromosome is the
  subject of intense research

33
Fitness Function

• A performance criterion that evaluates the
  performance of each individual phenotype. Higher
  is better.
• Examples object location, the closest robots to
  an object are selected maze navigation, those
  successfully navigating maze fastest are selected
• Choice of fitness function has consequence for
  artificial evolution
• The more detailed and constrained a fitness
  function is, the closer artificial evolution
  becomes to a supervised learning technique and
  less is left to emergence and autonomy of the
  evolving system

34
Selective Reproduction

• Copies are made of the best individuals in the
  population
• the probability of a given individual being
  reproduced equals its fitness divided by the sum
  fitness of the population
• Tournament Selection
• Elitism
• Copies are are then subjected to crossover with a
  random partner of the same generation, and
  mutation

35
(No Transcript)
36
(No Transcript)
37
Evolution of Simple Navigation
Fitness function has three components to be
maximized Speed, Straightness, Obstacle
Avoidance
38
(No Transcript)
39
(No Transcript)
40

• Robot reaches peak speed of 48 mm/s on
  straight-aways (max. speed is 80 mm/s)
• In terms of fitness function, there is no
  advantage for robots that move forward or
  backward. All robots move in direction of side
  with more sensors (front) thus maximizing
  information to deal with upcoming walls
• Both of these emerge from robot-environment
  interaction
• When compared to hand-coded robots, evolved
  robots performed better or comparable (as
  measured by fitness function)

41

• When the selection criterion changes (either a
  change in environment or fitness function), some
  individuals that previously were not among the
  best may be selected for reproduction and pull
  the population toward a new area of genetic space
• Thus, evolving systems are continuously adaptive
• Adaptation as displacement of a partially
  converged population in genetic space (Harvey
  1992, 1993)

42
Reactive Intelligence

• Sensors and motors are directly linked
• Agents react to the same sensory state with the
  same motor action

43
Active Perception

• What about cases where a robot must react
  differently to similar looking sensory patterns?
  (Perceptual aliasing problem)

44
Overcoming Perceptual Aliasing

• Agents partially determine the sensory patterns
  they receive from the environment by executing
  actions that modify the position of the agent
  with respect to the external environment or by
  altering the environment itself

45
Power and Limits of Reactive Agents

• Robots exploit the dimensions of the environment
• Performance decreases with increasingly radical
  changes of environment dimensions
• Reactive agents are able to exploit sensory-motor
  interactions and the environment to solve complex
  tasks and disambiguate similar percepts
• However, the agents limits are reached as the
  environment becomes increasingly variable and
  indeterministic

46
Modularity

• Modularity is an integral part of traditional
  functional decomposition approaches
• Specific behaviors allocated to different modules
• No mandate for modularity in evolutionary systems
• Is modularity beneficial for some tasks?
• Assuming we evolve modular controllers but let
  the evolutionary process determine the
  functionality of each module, will architectural
  modules correspond to basic behaviors?

47

• Modules do aid performance
• They do not map onto easily discernible functions
  from a distal perspective (outside looking in)
• Subsequent analysis shows that from a proximate
  perspective (from within), modules aid in
  producing different motor behaviors to similar
  sensory states
• Modules then are a straight-forward extension of
  the behavior of purely reactive agents

48
Hidden Layers

• Allow a re-representation of the input layer
• These re-representations may combine inputs from
  previous layer, like battery level and floor
  brightness, allowing behaviors to be based upon
  these new higher-level representations
• Spontaneous emergence of internal representations
  (crude topography map)

49
Lessons

• Increasing internal dynamics, e.g. modules or
  hidden layers, allows increased behavioral
  flexibility
• Behavioral flexibility allows for increased
  robustness in the face of environmental changes
• Environmental Generalization vs. Environmental
  Independence
• Abstraction results from overlapping domain
  representations at higher levels

50
Evolution and Learning

• Pro
• Learning allows individuals to adapt to changes
  in the environment that occur in the lifespan of
  an individual
• It can help and guide evolution
• Con
• Entails a delay in the ability to acquire fitness
• Increased unreliability
• Perhaps delayed reproduction

51

• Lamarkian Evolution
• Baldwin effect
• Evolution tends to select individuals who have
  already at birth those useful features which
  would otherwise be learned
• Indirect genetic assimilation, canalization

52

• Evolution can select for a predisposition to
  learn in a given domain. This predisposition may
  consist of
• The presence of starting conditions at birth,
  e.g. a particular architecture suitable for
  learning a certain task
• An inherited tendency to behave in such a way
  that the individual is exposed to the
  appropriately learning experiences

53
Competitive Co-evolution

• Co-evolution of competing populations (e.g.
  predator and prey) may produce increasingly
  complex evolving challenges
• May reciprocally drive one another to increasing
  levels of behavioral complexity by producing an
  evolutionary arms race
• May also result in cycling
• Co-evolving populations may cycle between
  alternative classes of strategies that do not
  represent progress in the long run, but are
  temporarily effective against the co-evolving
  population

54

• Co-evolution will result in increased behavioral
  complexity only if a general enough solution is
  found that is effective in a variety of
  environmental circumstances in order to avoid
  cycling
• This solution must
• Exist
• Be accessible to the agent on the genetic
  landscape

55
Conclusions

• Agents are embodied
• Agents are situated within an environment
• Agents often settle on solutions that are
  unintuitive, raising doubts about the efficacy of
  functional decomposition
• As the field develops, more interesting results
  will arise, for example, when more realistic
  implementations of genetic code are discovered

56
Thank You!

• Happy Thanksgiving