CAP6938 Neuroevolution and Developmental Encoding Non-Neural NEAT and Closing Remarks

1
CAP6938Neuroevolution and Developmental Encoding
Non-Neural NEAT andClosing Remarks

• Dr. Kenneth Stanley
• October 30, 2006

2
Outline

• Complexification is a general concept
• Protecting innovation is a general concept
• Therefore, they can apply to anything without a
  defined dimensionality
• Example Cellular Automata

3
Complexification is a General Concept

• Solving a smaller version of a problem and
  expanding the solution
• Making a rough estimate and refining it
• Building a structure piece by piece
• Elaboration of a pre-existing concept

4
Complexification Does Not Mean Optimizing Random
DimensionsFrom a Set

• Example 10-dimensional search space
• Now hold d2 through d10 constant and search d1
• Once you get a good value for d1, start searching
  both d1 and d2 together, and so on
• This is not complexification
• It is a naïve search assuming independent
  variables
• Subject to simple deception
• Usually wont work

5
Then What Does it Mean?

• Complexification means increasing information
  about the solution
• (Optimizing d1 does not increase general
  information about the solution)
• Initial dimensions are a complete solution on
  their own (nothing is held at zero)
• Complexification means finding the dimensionality
  of the solution is part of the problem
• A neural network can have any number of weight
  dimensions and solve the same problem
• Most dimensions outside the current structure
  have no meaning on their own

6
Example
1
3
2

• 3 dimensions
• Is dimension 2372 held at zero?
• What exactly is dimension 2372?
• It depends on how the other 2371 dimensions turn
  out to relate to each other
• It is undefined it doesnt exist
• Not like d10 in prior example, which always
  exists
• Complexification is searching infinite undefined
  dimensions, or rather, it is not performing
  search in the usual sense. It is increasing
  information.

7
Example 2
y
x

• Problem Find an expression of this function
• Complexification says start with a very low
  dimensional approximation as accurate as possible
  in its space
• Red line 2-dimensional estimate ymxb
• Now we could add new terms and refine the
  estimate
• Analogous to bending the line like a rubber band
  for each new dimension added
• New estimate does not necessarily need exactly
  the same term mxb

8
Protecting Innovation is a General Concept

• New ideas need time to mature
• Children need time to grow up
• Ph.D. students need room to make mistakes
• Bigger often means slower, but not stupider
• Einstein was not the teachers pet
• The long run is what matters
• If we kicked him out early, wed all lose

9
Speciation Protects Innovation

• An idea is represented as a niche
• The niche is a local, protected competition
• One niche does not directly compete with another
• Only the absolute worst are purged after
  sufficient opportunity is spent

10
General Concepts Means They Dont Have to Apply
to a Neural Network

• Complexification and protection of innovation go
  hand in hand
• In order to elaborate, one must protect potential
  elaborations
• In order to grow one must have room

11
Novel Phenotype Cellular Automata

• Set of pixels that change over time according to
  neighborhood rules
• The Game of Life is a familiar example of 2D
  cellular automata

From http//www.bitstorm.org/gameoflife/
12
2D Cellular Automata

• Pixels are in a line instead of a plane
• Change over time can be represented as a vertical
  graph

time
From Melanie Mitchell, James P. Crutchfield, and
Rajarshi Das, "Evolving Cellular Automata with
Genetic Algorithms A Review of Recent Work", In
Proceedings of the First International Conference
on Evolutionary Computation and Its Applications
(EvCA'96), Russian Academy of Sciences (1996).
13
Neighborhood Rules

• Next state for pixel determined by pixels in its
  neighborhood within some radius

2(2r1) bits per rule table
From Melanie Mitchell, James P. Crutchfield, and
Rajarshi Das, "Evolving Cellular Automata with
Genetic Algorithms A Review of Recent Work", In
Proceedings of the First International Conference
on Evolutionary Computation and Its Applications
(EvCA'96), Russian Academy of Sciences (1996).
14
Can It Do Anything Useful?

• Maybe it can compute functions
• Popular task Fill the line with whichever color
  is in the majority (Density Classification)
• Successful attempts (r3 128 bits/genome)

15
Assessing Performance

• Measure correct over unbiased distribution of
  many initial conditions
• Best performance is 86 on 149 pixels with r3
  (Juillé and Pollack 1998) using coevolution of
  rules and initial conditions
• Could we do a lot better than 86?
• Maybe with complexification

16
Complexifying Cellular Automata

• How?

17
Complexifying Cellular Automata

• How? Expand the neighborhood
• Neighborhood doesnt need to be symmetric or even
  contiguous
• Is this really complexification?
• Yes Unexpressed dimensions are undefined without
  knowing all the dimensions
• The initial rules give us only a little
  information, but good information
• The dimensions of the search space are the bits
  of the rule, not the neighborhood positions
• The rule includes the neighborhood positions,
  i.e. there is structure. Position is a
  historical marker in this case.

00 0
01 0
10 1
11 0
0 0
1 1
18
Even More Abstract Complexification Solution

• A very wide neighborhood could be input into a
  neural network that computes a function of those
  inputs and outputs the next value for the bit in
  the middle
• The network that computes the function can
  complexify

Evolved Topology



19
Conclusion

• Complexification and protection of innovation may
  allow more complex and therefore powerful
  neighborhood functions to evolve (maybe beat 86
  by using with coevolution?)
• Complexification and protection of innovation may
  allow far more complex solutions to anything

20
NE DE What Have We Learned

• Search is not just optimization
• Expanding complexity over generations is a
  powerful idea
• Protecting innovation is as well
• Neural networks can be grown with this method
• The mapping between genotype and phenotype is
  equally important
• Reuse of genes is powerful
• The neural model can be enhanced in several ways
• NEAT can evolve any kind of structure, including
  DE/indirect encodings, and CPPNs

21
What Is Its Significance?

• These are the forces of nature
• We are unlocking natures box by understanding
  the underlying algorithms
• Much of the beauty and complexity of nature, and
  civilization itself, resulted from these simple
  processes
• Biological evolution was an unguided process
• What will we create if we take its reigns and
  guide it?

22
Where is the Field?

• Two parallel streams
• How to evolve with complexification
• How to represent with reuse (DE)
• More progress so far on 1 than 2
• Indirect encoding is on the brink
• The two streams are merging (e.g. CPPNs)
• A complexifying system with an efficient encoding
  (mapping) is the next generation system

23
Next TopicsTechnical topics in implementing
complexifying evolutionary systems and presenting
results.

• Practical implementation issues
• Questions and group discussion/problem solving
• How to present research results

Reference Section 3 of Real-time Neuroevolution
in the NERO Video Game by Stanley, Bryant, and
Miikkulainen (2005). (Has most up-to-date NEAT
description)