AntNet: Distributed Stigmetric Control for Communications Networks

1
AntNet Distributed Stigmetric Control for
Communications Networks

• Gianni Di Caro Marco Dorigo
• Journal of Artificial Intelligence Research 1998
• Presentation by
• Tavaris Thomas

2
Presentation Contents

• Introduction/Background
• Model Description
• AntNet An Adaptive Agent-based Routing Algorithm
• Other Routing Algorithms
• Experimental Networks Used
• Results
• Conclusions and Future Work

3
Introduction/Background

• Increase in the supply and demand of network
  communication services
• Network Control online and off-line monitoring
  and management of the network resources
• Routing process or method of determining and
  prescribing incoming packets to an outgoing path
  (forwarding messages)

4
Swarm Intelligence (SI)

• New research field
• Collective behavior of social insects and other
  organisms
• ants, honey bees states/actions
• Stimergy Complex and intelligent behavior
  performed through the interaction of thousands of
  autonomous swarm members

5
Ant Colony Optimization(ACO)

• Foraging behavior of ants and is used
  successfully to solve combinatorial optimization
  problems.
• traveling salesman
• genome matching
• routing in telecommunications networks
• load balancing

6
Model Description

• WAN Irregular topology connection-less network
• Network communication is mapped on a directed
  weighted graph with N processing/forwarding nodes
• Links characterized by bandwidth (bit/sec) and
  transmission delay (sec)
• 2 types of packets (routing and data) routing
  have greater priority
• C based discrete event driven simulator

7
AntNet

• Adaptive, distributed, and mobile agent-based
  routing algorithm
• Reinforcement learning problems with hidden state
  (Bertsekas Tsitsiklis, 1996 Kaelbling,
  Littman, Moore, 1996 McCallum, 1995).

8
AntNet Algorithm Overview

• Mobile agents are asynchronously launched towards
  randomly selected destination nodes.
• Each agent searches for a minimum cost path
  joining its source and destination nodes.
• Each agent moves step-by-step towards its
  destination node. At each intermediate node a
  greedy stochastic policy is applied to choose the
  next node to move to. The policy makes use of (i)
  local agent-generated and maintained information,
  (ii) local problem-dependent heuristic
  information, and (iii) agent-private information.
• While moving, the agents collect information
  about the time length, the congestion status and
  the node identifiers of the followed path.

9
AntNet Algorithm Overview

• Once they have arrived at the destination, the
  agents go back to their source nodes by moving
  along the same path as before but in the opposite
  direction.
• During this backward travel, local models of the
  network status and the local routing table of
  each visited node are modified by the agents as a
  function of the path they followed and of its
  goodness.
• Once they have returned to their source node,
  the agents die.

10
Routing Table Contents
Goodness (desirability)
Routing table
Mean, variance, and best
Array of ds defining parametric statistical
model for the traffic distribution over the
network as seen by local node k
11
AntNet Algorithm

• The heuristic correction ln is a 0,1 normalized
  value proportional to the length qn (in bits
  waiting to be sent) of the queue of the link
  connecting the node k with its neighbor n
• The value of alpha weights the importance of the
  heuristic correction with respect to the
  probability values stored in the routing table.
  Agent's decisions are taken on the basis of a
  combination of a long-term learning process and
  an instantaneous heuristic prediction.
• Ideal alpha between 0.2 and 0.5

12
AntNet Algorithm

• The backward ant updates the routing table and
  arrays stored at each node as it propagates
  through network.

Positive reinforcement
Negative reinforcement
Reinforcement to be a function of the goodness
where
13
Other Routing Algorithms Compared

• OSPF (static, link state)Open Shortest Path First
• SPF (adaptive, link-state) Shortest Path First
• BF (adaptive, distance-vector) Bellman Ford
• Q-R (adaptive, distance-vector) Q-Routing
• PQ-R (adaptive, distance-vector) is the
  Predictive Q-Routing algorithm
• Daemon (adaptive, optimal routing) is an
  approximation of an ideal algorithm

14
Networks Used

• SimpleNet (1.9, 0.7, 8)

10Mbit/s and propagation delay of 1msec
mean shortest path distance, in terms of hops,
between all pairs of nodes, the variance Of
this average, and the total number of nodes
15
Networks Used

• NFSNET(2.2,0.8,14)

1.5Mbps propagation delays 4-20 msec
16
Networks Used

• NTTnet(6.5,3.8,57)
• 6Mbps propagation
• delay 1 to 5
• msec

17
Metrics for Performance Evaluation

• Throughput
• Delay Distribution- the authors used whole
  empirical distribution or to use the 90th
  percentile statistic, which allows one to compare
  the algorithms on the basis of the upper value of
  delay they were able to keep the 90 of the
  correctly delivered packets
• Network Capacity Usage (as expressed by the as
  the sum of the link capacities divided total
  available link capacity)

18
SimpleNet Throughput Results

• SimpleNet Comparison of algorithms for F-CBR
  traffic directed from node 1 to node 6)
• The delay distribution showed similar results
• note AntNet outperformed

19
NFSNET Delay Results

• Comparison of algorithms for increasing load for
  UP traffic. The load is increased reducing the
  MSIA (mean inter arrival time) value from 2.4 to
  2 seconds
• note that throughput results were similar
  amongst all algorithms but SPF and BF were the
  best

20
NTTnet Delay Results

• NTTnet Comparison of algorithms for increasing
  load for UP-HS traffic. The load is increased
  reducing the MSIA value from 4.1 to 3.7 seconds.
• note that throughput results were similar
  amongst all algorithms but SPF and BF were the
  best

21
Routing Overhead
Routing Overhead ratio between the bandwidth
occupied by the routing packets and the total
available network bandwidth. All data are scaled
by a factor of 10-3
22
Conclusions and Future Work

• AntNet showed superior performance and robustness
  to internal parameter settings for almost all the
  experiments.
• AntNet's most innovative aspect is the use of
  stigmetric communication to coordinate the
  actions of a set of agents that cooperate to
  build adaptive routing tables.

23
Future Work

• To add flow and error control to the algorithm
• Change the priority of ants as the propagate
  through the system
• Greater study of the negative reinforcement of
  connection
• Greater survivability in the presence of faults
  (disaster situations)