Loading...

PPT – Stochastic Learning Automata-Based Dynamic Algorithms for Single Source Shortest Path Problems PowerPoint presentation | free to download - id: 68f083-MTFlM

The Adobe Flash plugin is needed to view this content

Stochastic Learning Automata-Based Dynamic

Algorithms for Single Source Shortest Path

Problems

- S. Misra
- B. John Oommen
- Professor and Fellow of the IEEE
- Carleton University
- Ottawa
- Ontario, Canada
- (Nominated for the Best Paper Award)
- (Associated Thesis Proposal AAAI Doctoral

Award)

Outline

- Introduction
- Previous Dynamic Algorithms
- Ramalingam and Reps Algorithm
- Frigioni et al.s Algorithm
- Principles of Learning Automata (LA)
- Solution Model
- Proposed Algorithm
- Simulation and Experiments
- Conclusion

Dynamic Single Source Shortest Path Problem

(DSSSP)

- Maintaining shortest paths in a graph (with

single-source), where the edge-weights constantly

change, and where edges are constantly

inserted/deleted. - Edge-insertion is equivalent to weight-decrease,

and edge-deletion is equivalent to

weight-increase. - Semi-dynamic problem Either insertion

(weight-decrease) or deletion (weight-increase).

Dynamic Single Source Shortest Path Problem

(DSSSP)

- Fully-dynamic problem Both insertion

(weight-decrease) and deletion (weight-increase). - The problem is representative of many practical

situations in daily life. - What to do if the edge-weights keep changing? At

every time instant random edge-weights.. - How to get the SP for the average graph ? The

first known solution ???

Shortest Path Tree Costs are Random -

Changing...

Costs of the edges are determined on-the-fly Cost

BE 1.79 1.68 2.01, ..

The Static Algorithms

- Dijkstra or Bellman-Fords solutions are

unacceptably inefficient in dynamic environments. - Such static algorithms involve
- Recomputing the shortest path tree from scratch
- Done each time a topological change occurs.

Previous Dynamic Algorithms

- Spira and Pan (1975) Very early work, proven

theoretically to be inefficient. - McQuillan et al. (1980) Very early work, not

proven at all. - Ramalingam and Reps (1996) Recent work,

fully-dynamic solution. - Franciosa et al. (1997) Recent work,

semi-dynamic solution. - Frigioni et al. (2000) Recent work,

fully-dynamic solution.

Ramalingam and Reps Algorithm

- Edge-Insertion
- Maintains a priority queue containing vertices
- Priorities equal to their distance from the

end-point of the inserted edge. - When a vertex having a minimum priority is

extracted from the queue, all the outgoing edges

are processed. - Edge-Deletion
- Phase I Determines the vertices/edges affected

by the deletion. - Phase II Determines the new output value for all

the affected vertices and updates the shortest

path tree.

Frigioni et al.s Algorithm

- Weight-Decrease
- Based on Output updates.
- No. of Output Updates No. of vertices that

change the distance from the source, on unit

change in the graph. - If decreasing a weight changes the distance of

the terminating end of the inserted vertex, a

global priority queue is used to compute the new

distances from the source. - Unlike the Ramalingam/Reps algorithm, on

dequeuing a vertex, not all the edges leaving it

are scanned.

Frigioni et al.s Algorithm (Contd)

- Weight-Increase
- Based on the following node-coloring scheme
- Marking a node q white, which changes neither the

distance from s nor the parent in the tree rooted

in s - Marking a node q red, which increases the

distance from s, - marking a node q pink, which preserves its

distance from s, but replaces the old parent in

the tree rooted in s.

Frigioni et al.s Algorithm (Contd)

- Weight-Increase
- Three main phases
- Update local data-structures at the end-points of

the affected edge, and check whether any

distances change. - Color the vertices repeatedly by extracting

vertices with minimum priority. - Compute the new distances for the red vertices.

Learning Automata

- Previously used to model biological learning

systems. - Can be used to find the optimal action.
- How is learning accomplished?

Learning Automata The Feedback Loop

- Random Environment (RE)
- Learning Automata (LA)
- Set of actions ?1, ..., ?r
- Reward/Penalty
- Action Probability Vector
- Action Probability Updating Scheme The LRI

scheme.

? ?1, ..., ?r

? 0, 1

Variable Structure LA

? Defined in terms of ? Action Probability

Updating Schemes ? Action probability vector

is p1(t), ..., pr(t)T ? p i(t) Probability

of choosing ?i at time t ? Implemented using ?

random number generator ? Flexibility ?

Different actions at two consecutive time

instants ? Action probability Updated Various

ways

Categories of VSSA

- Classification based on the type of the

probability space - Continuous
- Discrete

- Classification based on the learning paradigm
- Reward-Penalty schemes
- Reward-Inaction schemes
- Inaction-Penalty schemes

Categories of VSSA

- Ergodic scheme
- ? Limiting Distribution Independent of initial

distribution - ? Used if Environment is Non-stationary
- ? LA wont get locked into any of the given

actions.

Categories of VSSA

- Absorbing scheme
- ? Limiting Distribution Dependent of initial

distribution - ? Used if Environment is Stationary
- ? LA finally gets Absorbed into its final action.

- Example Linear Reward-Inaction (LRI) scheme.

LRI Scheme

p1(n) p2(n) p3(n) p4(n)

0.4 0.3 0.1 0.2

If ?2 chosen rewarded. ? p2 increased ? p1,

p3, p4 decreased linearly.

p1(n1) p2(n1) p3(n1) p4(n1)

0.36 1-0.36-0.9-0.18 0.09 0.18

0.36 0.37 0.09 0.18

0 1 0 0

p1(?) p2(?) p3(?) p4(?)

If ?2 is the best action

Our Solution

- Current state-of-the-art No Solution to the

DSSSP - When the edge-weights are dynamically and

stochastically changing.

Our Solution

- Our solution
- Uses the Theory of Learning Automata (LA).
- Extends the current models by encapsulating the

problem within the field of LA. - Finds a shortest path in realistically occurring

stochastic environments. - Finds a shortest path for the average

underlying graph, dictated by an Oracle (also

called the Environment). - Finds the statistical shortest path tree, that

will be stable regardless of continuously

changing weights.

Our Solution Model The Automata

- Station a LA at every node in the graph.
- At every instance, the LA chooses a suitable edge

from all the outgoing edges at that node, by

interacting with the environment. - The LA requests the Environment for the current

random weight for the edge it chooses. - The system computes the current shortest path

using RR/FMN. - LA determines whether the choice it made should

be rewarded/penalized.

Our Solution Model The Environment

- Consists of the overall dynamically changing

graph. - Multiple edge-weights that change stochastically

and continuously. - Changes Based on a distribution
- Unknown to the LA,
- Known to the Environment.
- The Environment supplies a Reward/Penalty signal

to the LA.

Our Solution Model Reward/Penalty

- Updated shortest path tree is computed
- Based on the action the LA chooses, and
- The edge-weight the Environment provides.
- The LA compares the cost with the current

average shortest paths. - The LA
- Infers whether the choice should be

rewarded/penalized. - Updates the action probabilities using the LRI

scheme.

Shortest Path Tree Updated Action Probability

Vectors

4.9

LASPA The Proposed Algorithm

- INPUT
- G(V,E) A dynamically changing graph with

simultaneous multiple stochastic edge updates

occurring - iters total number of iterations.
- ? learning parameter.
- OUTPUT
- A converged graph that has all the shortest path

information. - Values of all action probability vectors.
- ASSUMPTION
- The algorithm maintains an action probability

vector, P p1(n), p2(n) pr(n), for each node

of the graph.

LASPA The Proposed Algorithm

- LASPA ALGORITHM
- 1 Obtain a snapshot of the directed graph with

each edge having a random weight. This

edge-weight is based on the random call for an

edge. - Run Dijkstras Algorithm to determine the

shortest path edges on the graphs snapshot

obtained in the first step. Based on this, update

the action probability vector of each node -

shortest path edges have an increased

probability. - Randomly choose a node from the current graph.

For that node, choose an edge based on the action

probability vector. Request the edge-weight of

this edge and recalculate the shortest path using

either the RR or FMN algorithms. - Update the action probability vectors for all the

nodes using the Reward-Inaction philosophy. - Repeat Steps 3-4 above until the algorithm has

converged.

Simulations The Experiments

- Experiment Set 1 Comparison of the performance

of LASPA with FMN and RR for a fixed graph

structure. - Experiment Set 2 Comparison of the performance

results with variation in graph structures. - Experiment Set 3 Sensitivity of the performance

of LASPA to the variation of certain parameters,

while keeping others constant.

Simulations The Performance Metrics

- Average number of scanned edges per update

operation. - Average number of processed nodes per update

operation - Average time required per update operation.

Simulations Experiment Set 1

- Average
- Processed Nodes
- Graph with 50 nodes, 20 sparsity.
- Edge-weights with means between 1.0 and 5.0, and

variances between 0.5 and 1.5 - Mixed sequences of 500 update operations.

Simulations Experiment Set 1 (Contd)

- Average
- Time Per Update
- Graph with 50 nodes, 20 sparsity.
- Edge-weights with means between 1.0 and 5.0, and

variances between 0.5 and 1.5 - Mixed sequences of 500 update operations.

Simulations Experiment Set 2

- Variation in
- Graph Sparsity
- Graphs with 100 nodes, varying sparsity.
- Edge-weights with means between 1.0 and 5.0, and

variances between 0.5 and 0.9, ?0.9 - Mixed sequences of 500 update operations.
- LASPA, on an average, performs better than

RR/FMN. - Variation in the number of nodes show similar

results.

Table Average/Min/Max Time Per Udate versus

Sparsity

Simulations Experiment Set 3

- Sensitivity of results
- to the variation in
- Learning parameter.
- Graph with 50 nodes, 20 sparsity.
- Edge-weights with means between 1.0 and 5.0, and

variances between 0.5 and 1.5 - Mixed sequences of 500 update operations.
- ? is varied.
- Other metrics show similar results.

Conclusions

- Novelty of our work
- First reported LA solution to DSSSP.
- Superior solution than the previous ones.
- Existing algorithms cant operate successfully in

realistically occurring continuously changing

stochastic environments. - Breakthrough solution that could have commercial

value. - Practical usefulness of our algorithm
- Telecommunications Networking
- Transportation
- Military
- Future work
- Evaluation on very large topologies.
- Evaluation on real networks.