Comparison of Q Routing and Shortest Path Algorithms

1
Comparison of Q Routing and Shortest Path
Algorithms

• Firat Tekiner, Z Ghassemlooy
• Optical Communications Research Group
• University of Northumbria, UK
• Srikanth Thadigol Reddappa
• Sheffield Hallam University, UK

2
overview

• Routing Algorithms
• Shortest Path Routing Algorithm
• Q Routing
• Dual Reinforcement Q Routing
• NSFNET
• PVM
• Simulation Results
• Conclusions

3
Routing

• Routing is the transmission of data (packets)
  from a source s to its destination d on the
  network or internetwork.
• 
  
  
  
• At every node the packet is received, stored and
  then routed to the next hop until it reaches its
  destination

4
Routing Algorithms

• Path Determination
• Computes the optimal path to every node in the
  network based on the routing metric of the links.
• Switching
• Forwards the packet to the next hop of the
  optimal path until it reaches destination

5
Classification

• Static Versus Dynamic
• Single-Path Versus Multipath
• Flat Versus Hierarchical
• Host-Intelligent Versus Router-Intelligent
• Intradomain Versus Interdomain
• Link-State Versus Distance Vector
• Adaptive Versus Non-Adaptive
• Adaptive Eg. Shortest Path Routing
• Non-Adaptive Eg. Q-learning methods, hybrid
  agent based Distance Vector algorithms

6
Shortest Path Algorithm

• Let
• V the array of all the edges (nodes)
• Dn the Array of Shortest cost of the
  vertices(links)
• Cs,w the cost of the vertices
• s Router Id
• w the neighbour
• Initialisation
• Dw 8
• w -gt(V-s) with minimal Cs, w
• Relax the node
• Dv min(Dv, Dwcw,v),
• where Cw,v are the costs of the vertices,
  connected to w
• Greedy Algorithm- Optimises the routes to all
  the nodes to a single least cost path and it
  insists to take this single best path without
  regard to future consequences.

7
Shortest Path Routing

• Computes Shortest Path using the Dijkstra
  Algorithm.
• Routes the packets based on least cost of the
  links
• Non-adaptive Routing Algorithm - routing table
  never change once initial routes have been
  selected unless there is a route failure
• Drawbacks
• Sometimes there may be congested queues in the
  intermediate nodes and packets spend waiting in
  the queues, which delays the packets travelling
  to the destination.
• For node r lying in the shortest path from s to
  d, it is also shortest path to r to s and d and
  vice versa

8
Q Routing

• Reinforcement learning all the nodes in the
  network learn a global routing policy
• Builds a routing table based on the delivery
  times (Q values) of the packets to every node in
  the network.
• Routes the packet to the destination on path with
  least delivery time
• For every data packet routed to the next hop, the
  node gets back a learning update about the
  remaining delivery time estimate for the packet
  to reach destination (Forward Exploration)
• Synchronised Q Information at every node in the
  network balances minimizing the number of
  hops a packet will take with the possibility of
  congestion along popular routes.

9
Forward Exploration

• When a node x sends a packet to node d via its
  neighbour y, it gets back ys estimated remaining
  trip times to the destination d, selects
• the neighbour with the smallest delivery time
• ?b is the learning rate parameter

10
Dual Reinforcement Q Routing

• Modified Q Routing with backward exploration
  along with backward exploration 2 way learning
• Every data packet carries the Q Information- the
  delivery time estimate of the sender node to its
  processor node (Backward Exploration)
• However adds overload on the packet

11
Backward Exploration

• When x sends a packet to node y to gets its
  estimated remaining trip times, y gets xs
  estimated trip times for its link with s.
• ?b is the learning rate parameter

12
NSFNET
13
PVM

• Software used for distributed computation
• Permits a heterogeneous collection of Unix and/or
  Windows computers hooked together by a network to
  be used as a single large parallel computer.
• Supports C, C , and Fortran languages
• Built in library functions
• Identifies the application as multiple tasks

14
Simulation

• Routing algorithms were implemented using PVM in
  C language on LINUX platform.
• Master and slave paradigm Every node of the
  NFSNET was assigned to a PVM task or a slave
  process, which is spawned by an initialising task
  or a master process on the same machine.
• 
  slave process
• Spawns
• 
  
• Single-program multiple-data Model
• Tasks with similar function executed in parallel

15
Simulation Conditions

• NSFNET is used as the network topology
• Random uniform traffic distribution is used with
  varying packet creation rates.
• For each simulation, packet generation is stopped
  after creating 5000 packets per node and
  simulation is stopped after all packets are
  arrived to their destinations or detected and
  deleted from the network.
• A random link fails every 5 seconds for a period
  of 4 seconds and traffic is directed to other
  neighbours.
• Hello packets were exchanged between the nodes to
  know whether its neighbouring nodes are active.
• Payload is fixed 512 Kbps

16
Performance Parameters

• Average packet delay
• Is the average delay a packet experiences
  while being routed from source to destination.
• Average throughput per packet
• Is the average number of packets being
  forwarded by a node for the duration of the
  simulation.

17
Average Packet Delay Vs. System Load
18
Throughput Vs. System Load
19
Throughput Vs. Learning Rate
20
Average Packet Delay vs. Learning Rate
21
Conclusions

• Q-Routing does not always guarantee to find the
  shortest path.
• At high system load QR display lower average
  packet delay compared with DRQR and SPRA
• Low bandwidth utilization in Q Routing and packet
  overhead in DRQR can be overcome by generating
  learning packets only when the queues get filled
  to a threshold
• Shortest path algorithms ignores the bottleneck
  as the network traffic increases.
• The need for an efficient adaptive routing
  algorithm.

22

• Thank you