ISE 410 Heuristics in Optimization Particle Swarm Optimization http://www.particleswarm.info/ http://www.swarmintelligence.org/

1
ISE 410 Heuristics in OptimizationParticle
Swarm Optimizationhttp//www.particleswarm.info/
http//www.swarmintelligence.org/
2
Swarm Intelligence

• Origins in Artificial Life (Alife) Research
• ALife studies how computational techniques can
  help when studying biological phenomena
• ALife studies how biological techniques can help
  out with computational problems
• Two main Swarm Intelligence based methods
• Particle Swarm Optimization (PSO)
• Ant Colony Optimization (ACO)

3
Swarm Intelligence

• Swarm Intelligence (SI) is the property of a
  system whereby
• the collective behaviors of (unsophisticated)
  agents
• interacting locally with their environment
• cause coherent functional global patterns to
  emerge.
• SI provides a basis with which it is possible to
  explore collective (or distributed) problem
  solving without centralized control or the
  provision of a global model.
• Leverage the power of complex adaptive systems to
  solve difficult non-linear stochastic problems

4
Swarm Intelligence

• Characteristics of a swarm
• Distributed, no central control or data source
• Limited communication
• No (explicit) model of the environment
• Perception of environment (sensing)
• Ability to react to environment changes.

5
Swarm Intelligence

• Social interactions (locally shared knowledge)
  provides the basis for unguided problem solving
• The efficiency of the effort is related to but
  not dependent upon the degree or connectedness of
  the network and the number of interacting agents

6
Swarm Intelligence

• Robust exemplars of problem-solving in Nature
• Survival in stochastic hostile environment
• Social interaction creates complex behaviors
• Behaviors modified by dynamic environment.
• Emergent behavior observed in
• Bacteria, immune system, ants, birds
• And other social animals

7
Particle Swarm Optimization(PSO)

• History
• Main idea and Algorithm
• Comparisons with GA
• Advantages and Disadvantages
• Implementation and Applications

8
Particle Swarm Optimization(PSO)

• History
• Main idea and Algorithm
• Comparisons with GA
• Advantages and Disadvantages
• Implementation and Applications

9
Origins and Inspiration of PSO

• Population based stochastic optimization
  technique inspired by social behaviour of bird
  flocking or fish schooling.
• Developed by Jim Kennedy, Bureau of Labor
  Statistics, U.S. Department of Labor and Russ
  Eberhart, Purdue University
• A concept for optimizing nonlinear functions
  using particle swarm methodology

10

• Inspired by simulation social behavior
• Related to bird flocking, fish schooling and
  swarming theory
• - steer toward the center
• - match neighbors velocity
• - avoid collisions
• Suppose
• a group of birds are randomly searching food in
  an area.
• There is only one piece of food in the area being
  searched.
• All the birds do not know where the food is. But
  they know how far the food is in each iteration.
• So what's the best strategy to find the food? The
  effective one is to follow the bird which is
  nearest to the food.

11
What is PSO?

• In PSO, each single solution is a "bird" in the
  search space.
• Call it "particle".
• All of particles have fitness values
• which are evaluated by the fitness function to be
  optimized, and
• have velocities
• which direct the flying of the particles.
• The particles fly through the problem space by
  following the current optimum particles.

12
PSO Algorithm

• Initialize with randomly generated particles.
• Update through generations in search for optima
• Each particle has a velocity and position
• Update for each particle uses two best values.
• Pbest best solution (fitness) it has achieved so
  far. (The fitness value is also stored.)
• Gbest best value, obtained so far by any
  particle in the population.

13

• PSO algorithm is not only a tool for
  optimization, but also a tool for representing
  sociocognition of human and artificial agents,
  based on principles of social psychology.
• A PSO system combines local search methods with
  global search methods, attempting to balance
  exploration and exploitation.

14

• Population-based search procedure in which
  individuals called particles change their
  position (state) with time.
• ? individual has position
• individual changes velocity

15

• Particles fly around in a multidimensional search
  space. During flight, each particle adjusts its
  position according to its own experience, and
  according to the experience of a neighboring
  particle, making use of the best position
  encountered by itself and its neighbor.

16
Particle Swarm Optimization (PSO) Process

• Initialize population in hyperspace
• Evaluate fitness of individual particles
• Modify velocities based on previous best and
  global (or neighborhood) best positions
• Terminate on some condition
• Go to step 2

17
PSO Algorithm

• Update each particle, each generation
• vi vi c1 rand() (pbesti -
  present) c2 rand() (gbesti -
  presenti) and
• presenti persenti vi
• where c1 and c2 are learning factors (weights)

a
b
18
PSO Algorithm
inertia
Personal influence
Social (global) influence

• Update each particle, each generation
• vi vi c1 rand() (pbesti -
  present) c2 rand() (gbesti -
  presenti) and
• presenti presenti vi
• where c1 and c2 are learning factors (weights)

a
b
19
PSO Algorithm

• Inertia Weight
• d is the dimension, c1 and c2 are positive
  constants, rand1 and rand2 are random numbers,
  and w is the inertia weight
• Velocity can be limited to Vmax

20
Particle Swarm Optimization(PSO)

• History
• Main idea and Algorithm
• Comparisons with GA
• Advantages and Disadvantages
• Implementation and Applications

21
PSO and GA Comparison

• Commonalities
• PSO and GA are both population based stochastic
  optimization
• both algorithms start with a group of a randomly
  generated population,
• both have fitness values to evaluate the
  population.
• Both update the population and search for the
  optimium with random techniques.
• Both systems do not guarantee success.

22
PSO and GA Comparison

• Differences
• PSO does not have genetic operators like
  crossover and mutation. Particles update
  themselves with the internal velocity.
• They also have memory, which is important to the
  algorithm.
• Particles do not die
• the information sharing mechanism in PSO is
  significantly different
• Info from best to others, GA population moves
  together

23

• PSO has a memory
• ?not what that best solution was, but where
  that best solution was
• Quality population responds to quality factors
  pbest and gbest
• Diverse response responses allocated between
  pbest and gbest
• Stability population changes state only when
  gbest changes
• Adaptability population does change state when
  gbest changes

24

• There is no selection in PSO
• ?all particles survive for the length of the run
• ?PSO is the only EA that does not remove
  candidate population members
• In PSO, topology is constant a neighbor is a
  neighbor
• Population size Jim 10-20, Russ 30-40

25
PSO Velocity Update Equations

• Global version vs Neighborhood version
• ? change pgd to pld .
• where pgd is the global best position
• and pld is the neighboring best
  position

26
Inertia Weight

• Large inertia weight facilitates global
  exploration, small on facilitates local
  exploration
• w must be selected carefully and/or decreased
  over the run
• Inertia weight seems to have attributes of
  temperature in simulated annealing

27
Vmax

• An important parameter in PSO typically the only
  one adjusted
• Clamps particles velocities on each dimension
• Determines fineness with which regions are
  searched
• ?if too high, can fly past optimal solutions
• ?if too low, can get stuck in local minima

28
PSO Pros and Cons

• Simple in concept
• Easy to implement
• Computationally efficient
• Application to combinatorial problems?
• ? Binary PSO

29
Books and Website

• Swarm Intelligence by Kennedy, Eberhart, and Shi,
  Morgan Kaufmann division of Academic Press, 2001.
• http//www.engr.iupui.edu/eberhart/web/PSOboo
  k.html
• http//www.particleswarm.net/
• http//web.ics.purdue.edu/hux/PSO.shtml
• http//www.cis.syr.edu/mohan/pso/
• http//clerc.maurice.free.fr/PSO/index.htm
• http//users.erols.com/cathyk/jimk.html

30
Ant Colony Optimization
31
ACO Concept

• Ants (blind) navigate from nest to food source
• Shortest path is discovered via pheromone trails
• each ant moves at random
• pheromone is deposited on path
• ants detect lead ants path, inclined to follow
• more pheromone on path increases probability of
  path being followed

32
ACO System

• Virtual trail accumulated on path segments
• Starting node selected at random
• Path selected at random
• based on amount of trail present on possible
  paths from starting node
• higher probability for paths with more trail
• Ant reaches next node, selects next path
• Continues until reaches starting node
• Finished tour is a solution

33
ACO System, cont.

• A completed tour is analyzed for optimality
• Trail amount adjusted to favor better solutions
• better solutions receive more trail
• worse solutions receive less trail
• higher probability of ant selecting path that is
  part of a better-performing tour
• New cycle is performed
• Repeated until most ants select the same tour on
  every cycle (convergence to solution)

34
ACO System, cont.

• Often applied to TSP (Travelling Salesman
  Problem) shortest path between n nodes
• Algorithm in Pseudocode
• Initialize Trail
• Do While (Stopping Criteria Not Satisfied)
  Cycle Loop
• Do Until (Each Ant Completes a Tour) Tour Loop
• Local Trail Update
• End Do
• Analyze Tours
• Global Trail Update
• End Do

35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
ACO Background

• Discrete optimization problems difficult to solve
• Soft computing techniques developed in past ten
  years
• Genetic algorithms (GAs)
• based on natural selection and genetics
• Ant Colony Optimization (ACO)
• modeling ant colony behavior

42
ACO Background, cont.

• Developed by Marco Dorigo (Milan, Italy), and
  others in early 1990s
• Some common applications
• Quadratic assignment problems
• Scheduling problems
• Dynamic routing problems in networks
• Theoretical analysis difficult
• algorithm is based on a series of random
  decisions (by artificial ants)
• probability of decisions changes on each
  iteration

43
What is ACO as Optimization Tech

• Probabilistic technique for solving computational
  problems which can be reduced to finding good
  paths through graphs
• They are inspired by the behavior of ants in
  finding paths from the colonyto food.

44
Implementation

• Can be used for both Static and Dynamic
  Combinatorial optimization problems
• Convergence is guaranteed, although the speed is
  unknown
• Value
• Solution

45
The Algorithm

• Ant Colony Algorithms are typically use to solve
  minimum cost problems.
• We may usually have N nodes and A undirected
  arcs
• There are two working modes for the ants either
  forwards or backwards.
• Pheromones are only deposited in backward mode.
  (so that we know how good the path was to update
  its trail)

46
The Algorithm

• The ants memory allows them to retrace the path
  it has followed while searching for the
  destination node
• Before moving backward on their memorized path,
  they eliminate any loops from it. While moving
  backwards, the ants leave pheromones on the arcs
  they traversed.

47
The Algorithm

• The ants evaluate the cost of the paths they have
  traversed.
• The shorter paths will receive a greater deposit
  of pheromones. An evaporation rule will be tied
  with the pheromones, which will reduce the chance
  for poor quality solutions.

48
The ACO Algorithm

• At the beginning of the search process, a
  constant amount of pheromone is assigned to all
  arcs. When located at a node i an ant k uses the
  pheromone trail to compute the probability of
  choosing j as the next node
• where is the neighborhood of ant k when in
  node i.

49
The Algorithm

• When the arc (i,j) is traversed , the pheromone
  value changes as follows
• By using this rule, the probability increases
  that forthcoming ants will use this arc.

50
The Algorithm

• After each ant k has moved to the next node, the
  pheromones evaporate by the following equation to
  all the arcs
• where is a parameter. An iteration
  is a complete cycle involving ants movement,
  pheromone evaporation, and pheromone deposit.

51
Steps for Solving a Problem by ACO

• Represent the problem in the form of sets of
  components and transitions, or by a set of
  weighted graphs, on which ants can build
  solutions
• Define the meaning of the pheromone trails
• Define the heuristic preference for the ant while
  constructing a solution
• If possible implement a efficient local search
  algorithm for the problem to be solved.
• Choose a specific ACO algorithm and apply to
  problem being solved
• Tune the parameter of the ACO algorithm.

52
Applications

• Efficiently Solves NP hard Problems
• Routing
• TSP (Traveling Salesman Problem)
• Vehicle Routing
• Sequential Ordering
• Assignment
• QAP (Quadratic Assignment Problem)
• Graph Coloring
• Generalized Assignment
• Frequency Assignment
• University Course Time Scheduling

53
Applications

• Scheduling
• Job Shop
• Open Shop
• Flow Shop
• Total tardiness (weighted/non-weighted)
• Project Scheduling
• Group Shop
• Subset
• Multi-Knapsack
• Max Independent Set
• Redundancy Allocation
• Set Covering
• Weight Constrained Graph Tree partition
• Arc-weighted L cardinality tree
• Maximum Clique

54
Applications

• Other
• Shortest Common Sequence
• Constraint Satisfaction
• 2D-HP protein folding
• Bin Packing
• Machine Learning
• Classification Rules
• Bayesian networks
• Fuzzy systems
• Network Routing
• Connection oriented network routing
• Connection network routing
• Optical network routing

55
Ant Colony Algorithms

• Let um and lm be the number of ants that have
  used the upper and lower branches.
• The probability Pu(m) with which the (m1)th ant
  chooses the upper branch is

56
Traveling Salesperson Problem

• Famous NP-Hard Optimization Problem
• Given a fully connected, symmetric G(V,E) with
  known edge costs, find the minimum cost tour.
• Artificial ants move from vertex to vertex to
  order to find the minimum cost tour using only
  pheromone mediated trails.

57
Traveling Salesperson Problem

• The three main ideas that this ant colony
  algorithm has adopted from real ant colonies are
  
• The ants have a probabilistic preference for
  paths with high pheromone value
• Shorter paths tend to have a higher rate of
  growth in pheromone value
• It uses an indirect communication system through
  pheromone in edges

58
Traveling Salesperson Problem

• Ants select the next vertex based on a weighted
  probability function based on two factors
• The number of edges and the associated cost
• The trail (pheromone) left behind by other ant
  agents.
• Each agent modifies the environment in two
  different ways
• Local trail updating As the ant moves between
  cities it updates the amount of pheromone on the
  edge
• Global trail updating When all ants have
  completed a tour the ant that found the shortest
  route updates the edges in its path

59
Traveling Salesperson Problem

• Local Updating is used to avoid very strong
  pheromone edges and hence increase exploration
  (and hopefully avoid locally optimal solutions).
• The Global Updating function gives the shortest
  path higher reinforcement by increasing the
  amount of pheromone on the edges of the shortest
  path.

60
Empirical Results

• Compared Ant Colony Algorithm to standard
  algorithms and meta-heuristic algorithms on
  Oliver 30 a 30 city TSP
• Standard 2-Opt, Lin-Kernighan,
• Meta-Heuristics Tabu Search and Simulated
  Annealing
• Conducted 10 replications of each algorithm and
  provided averaged results

61
Comparison to Standard Algorithms

• Examined Solution Quality not speed in
  general, standard algorithms were significantly
  faster.
• Best ACO solution - 420

62
Comparison to Meta-Heuristic Algorithms

• Meta-Heuristics are algorithms that can be
  applied to a variety of problems with a minimum
  of customization.
• Comparing ACO to other Meta-heuristics provides a
  fair market comparison (vice TSP specific
  algorithms).

63
Other Application Areas

• Scheduling Scheduling is a widespread problem
  of practical importance.
• Paul Forsyth Anthony Wren, University of Leeds
  Computer Science department developed a bus
  driver scheduling application using ant colony
  concepts.

64
Advantages and Disadvantages
65
Advantages and Disadvantages

• For TSPs (Traveling Salesman Problem), relatively
  efficient
• for a small number of nodes, TSPs can be solved
  by exhaustive search
• for a large number of nodes, TSPs are very
  computationally difficult to solve (NP-hard)
  exponential time to convergence
• Performs better against other global optimization
  techniques for TSP (neural net, genetic
  algorithms, simulated annealing)
• Compared to GAs (Genetic Algorithms)
• retains memory of entire colony instead of
  previous generation only
• less affected by poor initial solutions (due to
  combination of random path selection and colony
  memory)

66
Advantages and Disadvantages, cont.

• Can be used in dynamic applications (adapts to
  changes such as new distances, etc.)
• Has been applied to a wide variety of
  applications
• As with GAs, good choice for constrained discrete
  problems (not a gradient-based algorithm)

67
Advantages and Disadvantages, cont.

• Theoretical analysis is difficult
• Due to sequences of random decisions (not
  independent)
• Probability distribution changes by iteration
• Research is experimental rather than theoretical
• Convergence is guaranteed, but time to
  convergence uncertain

68
Advantages and Disadvantages, cont.

• Tradeoffs in evaluating convergence
• In NP-hard problems, need high-quality solutions
  quickly focus is on quality of solutions
• In dynamic network routing problems, need
  solutions for changing conditions focus is on
  effective evaluation of alternative paths
• Coding is somewhat complicated, not
  straightforward
• Pheromone trail additions/deletions, global
  updates and local updates
• Large number of different ACO algorithms to
  exploit different problem characteristics

69
Sources

• Dorigo, Marco and Stützle, Thomas. (2004) Ant
  Colony Optimization, Cambridge, MA The MIT
  Press.
• Dorigo, Marco, Gambardella, Luca M., Middendorf,
  Martin. (2002) Guest Editorial, IEEE
  Transactions on Evolutionary Computation, 6(4)
  317-320.
• Thompson, Jonathan, Ant Colony Optimization.
  http//www.orsoc.org.uk/region/regional/swords/swo
  rds.ppt, accessed April 24, 2005.
• Camp, Charles V., Bichon, Barron, J. and Stovall,
  Scott P. (2005) Design of Steel Frames Using
  Ant Colony Optimization, Journal of Structural
  Engineeering, 131 (3)369-379.
• Fjalldal, Johann Bragi, An Introduction to Ant
  Colony Algorithms. http//www.informatics.susse
  x.ac.uk/research/nlp/gazdar/teach/atc/1999/web/joh
  annf/ants.html, accessed April 24, 2005.

70
Questions?