Evolutionary Robotics

1
Evolutionary Robotics

• Tom Ziemke
• Dept. of Computer Science
• University of Skövde, Sweden
• tom_at_ida.his.se

2
Largely based on

• Nolfi Floreano (2000). Evolutionary Robotics
  The Biology, Intelligence and Technology of
  Self-Organizing Machines. MIT Press.

3
Evolutionary Robotics (ER)

• ER the attempt to develop robots and their
  sensorimotor control systems through an automatic
  design process ( self-organization) involving
  artificial evolution
• general procedure (as in all evolutionary
  computing)
• take an initial population of random individuals
• evaluate each individuals fitness
• let population reproduce using fitness-biased
  selection, crossover, mutation, etc.
• Go back to step ?

4
Why robotics is hard

• behavioral, physical systems are difficult to
  design
• much more than computer programs they depend on
  the interaction with their environment, which is
• often dynamic, unpredictable, etc.
• usually not fully accessible for the robot
• robot and environment form a dynamical system
• robots sensory state is a function of the
  environment and its own actions
• not easy to know a priori what internal
  mechanisms result in which behavior (and the
  other way round!)

5
ER vs. Robot Learning

• robot learning (e.g. using ANNs,) relies on the
  capacity to self-organize and generalize from a
  limited training set
• learning can be very useful where a (formal)
  model of the control task is lacking (e.g.
  ALVINN)
• but it requires explicit feedback
• targets in each time step in supervised learning
• occasional feedback in reinforcement learning
• ER shares reliance on self-organization
• but required amount of feedback (much) lower
• no constraints on what can be evolved

6
What is actually evolved?

• most common evolution of neural network weights
  as an alternative to conventional training
• because local (gradient descent) search methods
  have serious limitations
• in particular recurrent nets are difficult to
  train
• supervised and reinforcement learning require
  more a priori knowledge
• evolution of initial weights for life-time
  learning
• evolution of learning rules
• evolution of network architectures /
  modularization
• evolution of robot morphologies
• e.g. brain-body co-evolution

7
Khepera robot

• small size makes it easy to build and re-arrange
  environments (and relatively simple to simulate)

8
(No Transcript)
9
Example (1) - Looping maze
10
Looping maze experiment

• goals
• carried out entirely on the physical robot
• simple fitness function that emphasizes
  environmental interaction
• network
• eight inputs from infrared sensors
• two outputs control motors (1 forward, 0
  backward, 0.5 no motion)
• output layer is self-recurrent
• population
• 80 individuals (networks)
• fitness evaluation 80 steps ( 300 ms)

11
Fitness function

• goal
• robot should move fast, stay away from
  obstacles
• F V (1 - ?(?v)) (1 - I)
• V average rotation speed of the wheels
• maximized by speed
• ?v difference in speed
• maximized by straight motion
• i highest sensor activity (between 0 no object,
  1 touching object)
• maximized by distance from objects

12
Fitness curve (as usual)
13
Evolved direction of motion

• direction of motion not specified, but frontal
  direction emerges after a few generation

14

• state-space of the three components of the
  fitness function
• motion of evolution (equilibirum points of the
  best individuals)
• V 0.6
• 1-i 0.6 (i.e. i 0.4)
• 1 - ?(?v) 0.4

15
Example (2) - Homing

• additional sensors
• 2 light sensors
• one IR under the robot detects black and white
• battery (sim.)
• linear decrease in 50 cycles
• instantly recharged in the black area, near the
  light tower

16
(No Transcript)
17
Fitness function

• simpler version of the previous one
• F V (1 - i)
• V average rotation speed of the wheels
• maximized by speed
• i highest sensor activity (between 0 no object,
  1 touching object)
• maximized by distance from objects
• robot has to return to the zone, otherwise
  battery runs out, but it shouldnt stay there
  (fitness 0)

18
Evolutionary process

• population of 100 individuals
• each started with a full battery in each
  generation in a random position
• maximum of steps set to 150
• robot evolved for 10 days in a dark room (except
  for the light tower)
• after about 200 generations individuals were
  capable of
• navigating around the environment
• avoiding walls and the recharging area
• starting a homing trajectory when 1/3 of battery
  power left
• returning to the area when there were only 2-3
  battery-steps to go
• leaving immediately after recharge

19
(No Transcript)
20
Simulation vs. Reality

• controllers are often evolved in simulation
  because
• robot can damage itself or the environment when
  making mistakes
• evolution takes a lot of time
• many individuals many generations evaluation
  period
• transfer simulation ? reality
• possible if simulation is sufficiently realistic
  
• but that makes simulation more time-consuming
• evolved controllers are often evaluated or
  evolved further on the physical robot

21
Example (3) - Garbage-collecting robot

• 60 cm 35 cm environment, surrounded by 3cm
  walls
• equipped with 2 DOF gripper (up-down, open-close)
  
• task pick up target objects (height 3 cm
  diameter 2.3 cm)
• search for objects, avoiding walls
• recognize and approach objects
• pick up objects
• search for a wall, avoiding other objects
• recognize and approach wall
• release object

22
Example (4) - Competitive Co-Evolution

• CCE the evolution of two or more competing
  populations with coupled fitness
• e.g. predator - prey
• may enhance the power of artificial evolution
• evolutionary arms races competing populations
  can reciprocally drive each other to
  incrementally increasing levels of behavioral
  complexity

23
CCE Experiments with Kheperas
24
Body Brain Chicken Egg?

• Creating artificial life forms through
  evolutionary robotics faces a "chicken and egg"
  problem
• Learning to control a complex body is dominated
  by inductive biases specific to its sensors and
  effectors, while
• building a body which is controllable is
  conditioned on the pre-existence of a
  brain. (Funes Pollack, 1997)

25
Example (5) Experimental Setup
26
Evolving behavior and vision in predator and prey

• Preconditions
• Same maximum speed in both robots
• Speed in predator constrained by the view angle

27
Prey dominate
28
Variation Adding a constraint

• Preconditions
• Same maximum speed in both robots
• Speed in both robots constrained by the view
  angle

29
Predators dominate
30
Morphological space

• Prey choose speed over vision

31
Evolution of physical structures (Brandeis)

• E.g. Lego structures, first in simulation
• e.g. a bridge, fitness length from starting
  point

32
(No Transcript)
33
Golem Project - Brain-Body Co-Evolution

• In simulation and reality Golem project at
  Pollacks DEMO Lab, Brandeis (Lipson Pollack,
  2000)
• physical robot is (semi-) automatically
  constructed using 3D solid printing from
  thermoplastic material (only motors to be added)

34
Open issues in evolutionary robotics

• designers influence is still strong fitness
  function, genotype-phenotype mapping,
  environment, population sizes, robot body (number
  and position of sensors, etc.), control
  architecture, etc.
• but there is active research on all of those
  issues
• for applications time is certainly still a
  problem
• relevance for the understanding of natural
  systems
• ER models are extremely simplified
• but useful
• for understanding general principles (e.g.
  co-evolution, interaction between learning and
  evolution, etc.)
• fairly assumption-free modeling and hypothesis
  testing