Investigation of antnet routing algorithm by employing multiple ant colonies for packet switched net

1
Investigation of antnet routing algorithm by
employing multiple ant colonies for packet
switched networks to overcome the stagnation
problem

• Firat Tekiner (Phd Student)
• Z. Ghassemlooy
• Optical Communications Research Group, The
  University of Northumbria, Newcastle upon Tyne
• S. Alkhayatt
• Faculty of ACES, Sheffield Hallam University
• LCS 2004

2
Contents

• Background Information
• Reinforcement Learning
• Behaviour of ants in real life
• Antnet routing algorithm
• Antnet with Multiple Ant Colonies
• Antnet with Evaporation
• Simulation Environment and Results
• Concluding Remarks

3
Routing Problem

• In internetworking, the process of moving a
  packet of data from source to destination.
• A routing algorithm is necessary to find the
  optimal path (or the shortest path) from source
  to destination.
• Problems
• Existing algorithms are mostly Table-Based (high
  cost)
• Congestion and contention (requires traffic
  distribution)
• Requires human intelligence
• The routing algorithms that are in use are all
  static algorithms

4
Reinforcement Learning - Routing

• Q-Learning
• Q-routing (Boyan et al, 94) (Tekiner et al. 1,
  04)
• Dual reinforcement Q-routing (Kumar et al., 97
  01)
• Ant (software agent) based Routing Algorithms
• ABC routing (Schoonderwoerd et al., 96)
• Regular and Uniform ant routing (Subramanian et
  al., 97)
• Antnet (Dorigo et al., 98)
• Antnet (Dorigo et al., 02)
• Improved Antnet (Boyan et al., 02)
• Modified Antnet (Tekiner et al. 2, 04)
• Antnet with evaporation (Tekiner et al. 3, 04)
• Agent Distance Vector Routing (ADVR) (Amin et
  al., 01 02)

5
Comparison of Algorithms

• Antnet uses probabilistic routing tables whereas
  in well known Link State and Distance Vector
  algorithms routing table entries are
  deterministic
• Antnet uses less resources on the nodes
• Antnet is dynamic and self organising whereas
  Distance Vector and Link State algorithms require
  human supervision
• Q-Routing does not guarantee on finding the
  shortest path always. Moreover, they can only
  find a single path, they cannot explore multiple
  paths
• In antnet stagnation is the main problem (routing
  table freezes due to selecting same paths
  continously)

6
Ants In Nature - unsophisticated and simple

• Builds and protects their nests
• Sorts brood and food items
• Explore particular areas for food, and
  preferentially exploits the richest available
  food source
• Cooperates in carrying large items
• Migrates as colonies
• Leaves pheromones on their way back
• Stores information in the nature (uses world as a
  memory)
• Make decision in a stochastic way
• Always finds the shortest paths to their nests or
  food source
• Are blind, can not foresee future, and has very
  limited memory

7
Ants How do they find their way ?

• Notion of Stimergy Indirect communication via
  nature.
• Ants dont know where to go initially, and choose
  paths randomly
• Ants taking the shorter path will reach the
  destinations before the those taking a long
  route. The path is marked with pheromone.
• There after the number of ants using the shorter
  path will keep increasing, since more pheromone
  is laid on the path.

8
Ants in Antnet

• Software Agents (Ants) communicates with each
  other by using
• Probabilistic routing tables
• An array which represents statistical local
  traffic experienced by every node.
• Two types of software agents (ants)
• Forward Ants (collects information)
• Backward Ants (updates prob. table entries)
• Two types of queues
• Low priority queue (data packets and forward
  ants)
• High priority queue (backward ants and forward
  ants)

9
Antnet Algorithm Overview

• At regular intervals every node creates a forward
  ant to randomly selected destinations.
• Forward ants uses probabilistic routing tables
  together with queue status at every intermediate
  node to choose its output port from unvisited
  list of nodes.
• Time elapsed and node identifier is pushed to
  ants stack.
• If cycle is detected , cycle is deleted from ants
  memory.
• When a forward ant reaches to its destination
• It transforms itself to a backwad ant,
• Visits the list of the nodes in its stack in a
  reverse order,
• Updates corresponding entries in the routing
  tables and array on its way back to source by
  using its values stored on its stack.
• Ants reinforces the solution by the reinforcement
  parameter which is calculated by using trip times
  that it has experienced.

10
Antnet - Example
Forward - Ants Stack
Backward - Ants Stack
Lets assume that Pfd 14 and Pnd 7
Lets assume that Pfd 7 and Pnd 7
Routing Table for 3 Routing Table for 1
Routing Table for 0
11
Antnet With Multiple Ant Colonies
12
Antnet with Evaporation

• Evaporation is a real life scenario where
  pheromone laid by real ants evaporates in time
  due to natural circumstances.
• Link usage statistics are used to evaporate
  (e(x)) the phernome laid by the ants. It is the
  proportion of number of forward ants destined to
  the node x over the total ants received by the
  current node in the given time window.
• Frequency of evaporation is defined by the
  programmer.
• Amount of probability to be evaporated is
  subtracted from the associated link. Then this
  amount is equally distributed over the other
  links.

13
Simulation Parameters

• Parallel Virtual Machine (PVM) together with C
  Language is used to simulate the algorithm
• Every network node is assigned to a different
  process
• Poisson traffic distribution
• 2500 packets created per node
• Average of 15 simulation runs is used for
  accuracy
• No packet loss due to node/link failures
• Ant creation rate is set to 5sec per node and
  Evaporation rate is set to 0.5 sec per link.
• 29 Node Random Network is used.

14
Results Table

• Antnet with multiple ant colonies performed the
  best in terms of throughput.
• Antnet with evaporation performed best in terms
  of average packet delay.

15
Results - Throughput
Throughput vs. Simulation Time
16
Concluding Remarks

• Stagnation is a major problem but solution
  exists.
• Multiple Ant Colonies applied to the antnet
  routing algorithm for the packet switched
  networks.
• Multiple Ant Colonies has increased the
  throughput of the network whereas there is no
  improvement observed on the average delay
  experienced per packet.
• No interaction among the different colonies has
  been considered. This can be taken into account
  in the future.
• By evaporating the links probabilities in a
  predefined rate, average delay experienced per
  packet is reduced by 7.
• No mathematical formula only constant variables
  are used in antnet.
• There is a need for a second meta-heruistic to
  optimise antnets parameters

17
Thanks!

• Any Questions?