# Stochastic Learning Automata-Based Dynamic Algorithms for Single Source Shortest Path Problems - PowerPoint PPT Presentation

PPT – Stochastic Learning Automata-Based Dynamic Algorithms for Single Source Shortest Path Problems PowerPoint presentation | free to download - id: 68f083-MTFlM The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## Stochastic Learning Automata-Based Dynamic Algorithms for Single Source Shortest Path Problems

Description:

### Stochastic Learning Automata-Based Dynamic Algorithms for Single Source Shortest Path Problems S. Misra B. John Oommen Professor and Fellow of the IEEE – PowerPoint PPT presentation

Number of Views:5
Avg rating:3.0/5.0
Slides: 34
Provided by: peopleScs5
Category:
Tags:
Transcript and Presenter's Notes

Title: Stochastic Learning Automata-Based Dynamic Algorithms for Single Source Shortest Path Problems

1
Stochastic Learning Automata-Based Dynamic
Algorithms for Single Source Shortest Path
Problems
• S. Misra
• B. John Oommen
• Professor and Fellow of the IEEE
• Carleton University
• Ottawa
• (Nominated for the Best Paper Award)
• (Associated Thesis Proposal AAAI Doctoral
Award)

2
Outline
• Introduction
• Previous Dynamic Algorithms
• Ramalingam and Reps Algorithm
• Frigioni et al.s Algorithm
• Principles of Learning Automata (LA)
• Solution Model
• Proposed Algorithm
• Simulation and Experiments
• Conclusion

3
Dynamic Single Source Shortest Path Problem
(DSSSP)
• Maintaining shortest paths in a graph (with
single-source), where the edge-weights constantly
change, and where edges are constantly
inserted/deleted.
• Edge-insertion is equivalent to weight-decrease,
and edge-deletion is equivalent to
weight-increase.
• Semi-dynamic problem Either insertion
(weight-decrease) or deletion (weight-increase).

4
Dynamic Single Source Shortest Path Problem
(DSSSP)
• Fully-dynamic problem Both insertion
(weight-decrease) and deletion (weight-increase).
• The problem is representative of many practical
situations in daily life.
• What to do if the edge-weights keep changing? At
every time instant random edge-weights..
• How to get the SP for the average graph ? The
first known solution ???

5
Shortest Path Tree Costs are Random -
Changing...
Costs of the edges are determined on-the-fly Cost
BE 1.79 1.68 2.01, ..
6
The Static Algorithms
• Dijkstra or Bellman-Fords solutions are
unacceptably inefficient in dynamic environments.
• Such static algorithms involve
• Recomputing the shortest path tree from scratch
• Done each time a topological change occurs.

7
Previous Dynamic Algorithms
• Spira and Pan (1975) Very early work, proven
theoretically to be inefficient.
• McQuillan et al. (1980) Very early work, not
proven at all.
• Ramalingam and Reps (1996) Recent work,
fully-dynamic solution.
• Franciosa et al. (1997) Recent work,
semi-dynamic solution.
• Frigioni et al. (2000) Recent work,
fully-dynamic solution.

8
Ramalingam and Reps Algorithm
• Edge-Insertion
• Maintains a priority queue containing vertices
• Priorities equal to their distance from the
end-point of the inserted edge.
• When a vertex having a minimum priority is
extracted from the queue, all the outgoing edges
are processed.
• Edge-Deletion
• Phase I Determines the vertices/edges affected
by the deletion.
• Phase II Determines the new output value for all
the affected vertices and updates the shortest
path tree.

9
Frigioni et al.s Algorithm
• Weight-Decrease
• No. of Output Updates No. of vertices that
change the distance from the source, on unit
change in the graph.
• If decreasing a weight changes the distance of
the terminating end of the inserted vertex, a
global priority queue is used to compute the new
distances from the source.
• Unlike the Ramalingam/Reps algorithm, on
dequeuing a vertex, not all the edges leaving it
are scanned.

10
Frigioni et al.s Algorithm (Contd)
• Weight-Increase
• Based on the following node-coloring scheme
• Marking a node q white, which changes neither the
distance from s nor the parent in the tree rooted
in s
• Marking a node q red, which increases the
distance from s,
• marking a node q pink, which preserves its
distance from s, but replaces the old parent in
the tree rooted in s.

11
Frigioni et al.s Algorithm (Contd)
• Weight-Increase
• Three main phases
• Update local data-structures at the end-points of
the affected edge, and check whether any
distances change.
• Color the vertices repeatedly by extracting
vertices with minimum priority.
• Compute the new distances for the red vertices.

12
Learning Automata
• Previously used to model biological learning
systems.
• Can be used to find the optimal action.
• How is learning accomplished?

13
Learning Automata The Feedback Loop
• Random Environment (RE)
• Learning Automata (LA)
• Set of actions ?1, ..., ?r
• Reward/Penalty
• Action Probability Vector
• Action Probability Updating Scheme The LRI
scheme.

? ?1, ..., ?r
? 0, 1
14
Variable Structure LA
? Defined in terms of ? Action Probability
Updating Schemes ? Action probability vector
is p1(t), ..., pr(t)T ? p i(t) Probability
of choosing ?i at time t ? Implemented using ?
random number generator ? Flexibility ?
Different actions at two consecutive time
instants ? Action probability Updated Various
ways
15
Categories of VSSA
• Classification based on the type of the
probability space
• Continuous
• Discrete
• Classification based on the learning paradigm
• Reward-Penalty schemes
• Reward-Inaction schemes
• Inaction-Penalty schemes

16
Categories of VSSA
• Ergodic scheme
• ? Limiting Distribution Independent of initial
distribution
• ? Used if Environment is Non-stationary
• ? LA wont get locked into any of the given
actions.

17
Categories of VSSA
• Absorbing scheme
• ? Limiting Distribution Dependent of initial
distribution
• ? Used if Environment is Stationary
• ? LA finally gets Absorbed into its final action.
• Example Linear Reward-Inaction (LRI) scheme.

18
LRI Scheme
p1(n) p2(n) p3(n) p4(n)
0.4 0.3 0.1 0.2
If ?2 chosen rewarded. ? p2 increased ? p1,
p3, p4 decreased linearly.

p1(n1) p2(n1) p3(n1) p4(n1)
0.36 1-0.36-0.9-0.18 0.09 0.18
0.36 0.37 0.09 0.18

0 1 0 0
p1(?) p2(?) p3(?) p4(?)
If ?2 is the best action
19
Our Solution
• Current state-of-the-art No Solution to the
DSSSP
• When the edge-weights are dynamically and
stochastically changing.

20
Our Solution
• Our solution
• Uses the Theory of Learning Automata (LA).
• Extends the current models by encapsulating the
problem within the field of LA.
• Finds a shortest path in realistically occurring
stochastic environments.
• Finds a shortest path for the average
underlying graph, dictated by an Oracle (also
called the Environment).
• Finds the statistical shortest path tree, that
will be stable regardless of continuously
changing weights.

21
Our Solution Model The Automata
• Station a LA at every node in the graph.
• At every instance, the LA chooses a suitable edge
from all the outgoing edges at that node, by
interacting with the environment.
• The LA requests the Environment for the current
random weight for the edge it chooses.
• The system computes the current shortest path
using RR/FMN.
• LA determines whether the choice it made should
be rewarded/penalized.

22
Our Solution Model The Environment
• Consists of the overall dynamically changing
graph.
• Multiple edge-weights that change stochastically
and continuously.
• Changes Based on a distribution
• Unknown to the LA,
• Known to the Environment.
• The Environment supplies a Reward/Penalty signal
to the LA.

23
Our Solution Model Reward/Penalty
• Updated shortest path tree is computed
• Based on the action the LA chooses, and
• The edge-weight the Environment provides.
• The LA compares the cost with the current
average shortest paths.
• The LA
• Infers whether the choice should be
rewarded/penalized.
• Updates the action probabilities using the LRI
scheme.

24
Shortest Path Tree Updated Action Probability
Vectors
4.9
25
LASPA The Proposed Algorithm
• INPUT
• G(V,E) A dynamically changing graph with
occurring
• iters total number of iterations.
• ? learning parameter.
• OUTPUT
• A converged graph that has all the shortest path
information.
• Values of all action probability vectors.
• ASSUMPTION
• The algorithm maintains an action probability
vector, P p1(n), p2(n) pr(n), for each node
of the graph.

26
LASPA The Proposed Algorithm
• LASPA ALGORITHM
• 1 Obtain a snapshot of the directed graph with
each edge having a random weight. This
edge-weight is based on the random call for an
edge.
• Run Dijkstras Algorithm to determine the
shortest path edges on the graphs snapshot
obtained in the first step. Based on this, update
the action probability vector of each node -
shortest path edges have an increased
probability.
• Randomly choose a node from the current graph.
For that node, choose an edge based on the action
probability vector. Request the edge-weight of
this edge and recalculate the shortest path using
either the RR or FMN algorithms.
• Update the action probability vectors for all the
nodes using the Reward-Inaction philosophy.
• Repeat Steps 3-4 above until the algorithm has
converged.

27
Simulations The Experiments
• Experiment Set 1 Comparison of the performance
of LASPA with FMN and RR for a fixed graph
structure.
• Experiment Set 2 Comparison of the performance
results with variation in graph structures.
• Experiment Set 3 Sensitivity of the performance
of LASPA to the variation of certain parameters,
while keeping others constant.

28
Simulations The Performance Metrics
• Average number of scanned edges per update
operation.
• Average number of processed nodes per update
operation
• Average time required per update operation.

29
Simulations Experiment Set 1
• Average
• Processed Nodes
• Graph with 50 nodes, 20 sparsity.
• Edge-weights with means between 1.0 and 5.0, and
variances between 0.5 and 1.5
• Mixed sequences of 500 update operations.

30
Simulations Experiment Set 1 (Contd)
• Average
• Time Per Update
• Graph with 50 nodes, 20 sparsity.
• Edge-weights with means between 1.0 and 5.0, and
variances between 0.5 and 1.5
• Mixed sequences of 500 update operations.

31
Simulations Experiment Set 2
• Variation in
• Graph Sparsity
• Graphs with 100 nodes, varying sparsity.
• Edge-weights with means between 1.0 and 5.0, and
variances between 0.5 and 0.9, ?0.9
• Mixed sequences of 500 update operations.
• LASPA, on an average, performs better than
RR/FMN.
• Variation in the number of nodes show similar
results.

Table Average/Min/Max Time Per Udate versus
Sparsity
32
Simulations Experiment Set 3
• Sensitivity of results
• to the variation in
• Learning parameter.
• Graph with 50 nodes, 20 sparsity.
• Edge-weights with means between 1.0 and 5.0, and
variances between 0.5 and 1.5
• Mixed sequences of 500 update operations.
• ? is varied.
• Other metrics show similar results.

33
Conclusions
• Novelty of our work
• First reported LA solution to DSSSP.
• Superior solution than the previous ones.
• Existing algorithms cant operate successfully in
realistically occurring continuously changing
stochastic environments.
• Breakthrough solution that could have commercial
value.
• Practical usefulness of our algorithm
• Telecommunications Networking
• Transportation
• Military
• Future work
• Evaluation on very large topologies.
• Evaluation on real networks.