The Life Cycle Model: Combining Particle Swarm Optimization, Genetic Algorithms, and Stochastic Hill Climbers

1
The Life Cycle Model Combining Particle Swarm
Optimization, Genetic Algorithms, and Stochastic
Hill Climbers
By Thiemo Krink and Morten Lovbjerg

• Romerl Elizes
• DCS891C Presentation

2
Agenda

• What is the Life Cycle Model?
• Definitions
• Research methodology
• Research results
• Related work
• Personal observations
• Future work

3
Life Cycle Model

• What is Life Cycle?
• Based on Biological life cycle
• Goal optimal solution

4
Why Life Cycle Model?

• Life Cycle Model is used to provide a
  comprehensive search heuristic that includes
• Particle swarm optimization
• Genetic algorithms
• Stochastic hill climber
• LC Model criteria
• Each LC test function started out with an element
  using PSO.
• If no appreciable optimization occurs after 50
  iterations, element is optimized in GA.
• If no appreciable optimization occurs after 50
  iterations, element is optimized in SHC.
• Process continues until an optimization point is
  reached or end-cycle condition is reached.
• Life Cycle Model is better than the three
  algorithms separately.

5
Particle Swarm Optimization (PSO)

• Definition swarm of insects or school of fish
  honing in on a desirable location.
• Variables position and velocity. Updated at each
  iteration to reach an optimal solution.
• Strengths best suited for numerical
  optimization, cooperation, competition.
• Weaknesses convergences on local optima

6
Genetic Algorithms (GA)

• Definition two populations of individuals are
  merged together to arrive at a new population
  with the best characteristics of the two
  populations.
• Variable any desirable trait to induce mutation
  or crossover.
• Strengths more comprehensive and scrutinizing,
  cooperation, competition.
• Weakness converges on local optima, convergence
  starts early

7
Stochastic Hill Climbing (SHC)

• Definition given some randomly assigned variable
  (node), determine if the nearest node has better
  characteristics.
• Variable some node.
• Strengths good for local search and finding
  closest optimum.
• Weaknesses performance dependent on starting
  variable, relegated only to local optima.

8
Research Methodology

• 5 different quantitative test functions were run
  against the PSO, SHC, GA, and LC Models.
• 5 Test functions were Sphere, Rosenbrock,
  Griewank, Rastrigin, and Ackley. Sphere was the
  simplest x2
• PSO tests used 20 individuals
• HC tests used 25 individuals
• GA tests used 100 individuals
• LC tests used 150 individuals

9
Research Results

• Main test variables are Fitness vs. 2.5 million
  Evaluations
• Fitness criteria lowest number in the experiment
  is optimum

PSO GA HC LC
Sphere 1 3 4 2
Rosenbrock 1 3 4 2
Griewank 2 3 4 1
Rastrigin 3 1 4 2
Ackley 3 2 4 1
Weighted
Average 2 2.4 4 1.6
10
Research Results

• Final composition of elements after 2.5 million
  evaluations

Sphere GA
Rosenbrock GA
Griewank PSO
Rastrigin GA
Ackley PSO
11
Related Work

• C. Grosan, A. Abraham, M. Nicoara. Performance
  Tuning of Evolutionary Algorithms Using Particle
  Sub Swarms. IEEE Seventh International Symposium
  on Symbolic and Numeric Algorithms for Scientific
  Computing (SYNASC, 2005) pp. 287-294.
  Evolutionary Algorithms and Particle Swarm
  Optimization for Mathematical Problems.
• R. Senaratne, S. Halgamuge. Optimized Landmark
  Model Matching for Face Recognition. IEEE Seventh
  International Conference on Automatic Face and
  Gesture Recognition (FGR, 2006). Landmark Model
  Matching, Elastic Bunch Graph Matching and Active
  Shape Model, and Particle Swarm Optimization for
  Face Recognition.
• C. Nunes, A. Britto, C. Kaestner, R. Sabourin. An
  Optimized Hill Climbing Algorithm for Feature
  Subset Selection Evaluation on Handwritten
  Character Recognition. IEEE Ninth International
  Workshop on Frontiers in Handwriting Recognition
  (IWFHR, 2004). Optimized Hill Climbing Algorithm
  for Handwritten Character Recognition.

12
Personal Observations

• Hill Climbing should be taken out of the equation
  for this experiment.
• Current work took into account convergence
  factors for individual algorithms.
• Why PSO as start object? Why not others?

13
Future Work

• Experiment with different start objects.
• Add more test functions and use score card to
  evaluate these algorithms against these test
  functions.
• Add more criteria to critically evaluate
  algorithms time, (O) time complexity, and
  importance of convergence.
• Life Cycle Model to extend to possibly include
  Evolutionary Algorithms, Repulsive Particle Swarm
  Optimization, and Ant Colony Optimization.
• Apply the Life Cycle Model to the Character
  Recognition and Face Recognition research
• Other research possibilities code breaking,
  learning robot behavior, electronic circuit
  design, molecular structure optimization, mobile
  communications optimizations, plant floor layout,
  tactical asset allocation, and international
  equity strategies.