Shortest Paths

1
Shortest Paths

• Definitions
• Single Source Algorithms
• Bellman Ford
• DAG shortest path algorithm
• Dijkstra
• All Pairs Algorithms
• Using Single Source Algorithms
• Matrix multiplication
• Floyd-Warshall
• Both of above use adjacency matrix representation
  and dynamic programming
• Johnsons algorithm
• Uses adjacency list representation

2
Single Source Definition

• Input
• Weighted, connected directed graph G(V,E)
• Weight (length) function w on each edge e in E
• Source node s in V
• Task
• Compute a shortest path from s to all nodes in V

3
All Pairs Definition

• Input
• Weighted, connected directed graph G(V,E)
• Weight (length) function w on each edge e in E
• We will typically assume w is represented as a
  matrix
• Task
• Compute a shortest path from all nodes in V to
  all nodes in V

4
Comments

• If edges are not weighted, then BFS works for
  single source problem
• Optimal substructure
• A shortest path between s and t contains other
  shortest paths within it
• No known algorithm is better at finding a
  shortest path from s to a specific destination
  node t in G than finding the shortest path from s
  to all nodes in V

5
Negative weight edges

• Negative weight edges can be allowed as long as
  there are no negative weight cycles
• If there are negative weight cycles, then there
  cannot be a shortest path from s to any node t
  (why?)
• If we disallow negative weight cycles, then there
  always is a shortest path that contains no cycles

6
Relaxation technique

• For each vertex v, we maintain an upper bound
  dv on the weight of shortest path from s to v
• dv initialized to infinity
• Relaxing an edge (u,v)
• Can we improve the shortest path to v by going
  through u?
• If dv gt du w(u,v), dv du w(u,v)
• This can be done in O(1) time

7
Bellman-Ford Algorithm

• Bellman-Ford (G, w, s)
• Initialize-Single-Source(G,s)
• for (i1 to V-1)
• for each edge (u,v) in E
• relax(u,v)
• for each edge (u,v) in E
• if dv gt du w(u,v)
• return NEGATIVE WEIGHT CYCLE

8
Running Time

• for (i1 to V-1)
• for each edge (u,v) in E
• relax(u,v)
• The above takes (V-1)O(E) O(VE) time
• for each edge (u,v) in E
• if dv gt du w(u,v)
• return NEGATIVE WEIGHT CYCLE
• The above takes O(E) time

9
Proof of Correctness

• If there is a shortest path from s to any node v,
  then dv will have this weight at end
• Let p (e1, e2, , ek) (v1, v2, v3, , v,1)
  be a shortest path from s to v
• s v1, v vk1, ei (vi, vi1)
• At beginning of ith iteration, during which we
  relax edge ei, dvi must be correct, so at end
  of ith iteration, dvi1 will e correct
• Iteration is the full sweep of relaxing all edges
• Proof by induction

10
Negative weight cycle

• for each edge (u,v) in E
• if dv gt du w(u,v)
• return NEGATIVE WEIGHT CYCLE
• If no neg weight cycle, dv du w(u,v) for
  all (u,v)
• If there is a negative weight cycle C, for some
  edge (u,v) on C, it must be the case that dv gt
  du w(u,v).
• Suppose this is not true for some neg. weight
  cycle C
• sum these (dv du w(u,v)) all the way
  around C
• We end up with sum(dv) sum(du) weight(C)
• This is impossible unless weight(C) 0
• But weight(C) is negative, so this cannot happen
• Thus for some (u,v) on C, dv gt du w(u,v)

11
DAG shortest path algorithm
5

• DAG-SP (G, w, s)
• Initialize-Single-Source(G,s)
• Topologically sort vertices in G
• for each vertex u, taken in sorted order
• for each edge (u,v) in E
• relax(u,v)

3
4
-4
s
5
1
3
8
12
Running time Improvement

• O(VE) for the topological sorting
• We only do 1 relaxation for each edge O(E) time
• for each vertex u, taken in sorted order
• for each edge (u,v) in E
• relax(u,v)
• Overall running time O(VE)

13
Proof of Correctness

• If there is a shortest path from s to any node v,
  then dv will have this weight at end
• Let p (e1, e2, , ek) (v1, v2, v3, , v,1)
  be a shortest path from s to v
• s v1, v vk1, ei (vi, vi1)
• Since we sort edges in topological order, we will
  process node vi (and edge ei) before processing
  later edges in the path.

14
Dijkstras Algorithm

• Dijkstra (G, w, s)
• / Assumption all edge weights non-negative /
• Initialize-Single-Source(G,s)
• Completed
• ToBeCompleted V
• While ToBeCompleted is not empty
• u EXTRACT-MIN(ToBeCompleted)
• Completed u
• for each edge (u,v) relax(u,v)

15
Running Time Analysis

• While ToBeCompleted is not empty
• u EXTRACT-MIN(ToBeCompleted)
• Completed u
• for each edge (u,v) relax(u,v)
• Each edge relaxed at most once O(E)
• Need to decrease-key potentially once per edge
• Need to extract-min once per node

16
Running Time Analysis contd

• O(E) edge relaxations
• Priority Queue operations
• O(E) decrease key operations
• O(V) extract-min operations
• Three implementations of priority queues
• Array O(V2) time
• decrease-key is O(1) and extract-min is O(V)
• Binary heap O(E log V) time assuming E gt V
• decrease-key and extract-min are O(log V)
• Fibonacci heap O(V log V E) time
• decrease-key is O(1) amortized time and
  extract-min is O(log V)

17
Compare Dijkstras algorithm to Prims algorithm
for MST

• Dijsktra
• Priority Queue operations
• O(E) decrease key operations
• O(V) extract-min operations
• Prim
• Priority Queue operations
• O(E) decrease-key operations
• O(V) extract-min operations
• Is this a coincidence or is there something more
  here?

18
Proof of Correctness

• Assume that Dijkstras algorithm fails to compute
  length of all shortest paths from s
• Let v be the first node whose shortest path
  length is computed incorrectly
• Let S be the set of nodes whose shortest paths
  were computed correctly by Dijkstra prior to
  adding v to the processed set of nodes.
• Dijkstras algorithm has used the shortest path
  from s to v using only nodes in S when it added v
  to S.
• The shortest path to v must include one node not
  in S
• Let u be the first such node.

19
Proof of Correctness

• The length of the shortest path to u must be at
  least that of the length of the path computed to
  v.
• Why?
• The length of the path from u to v must be lt 0.
• Why?
• No path can have negative length since all edge
  weights are non-negative, and thus we have a
  contradiction.

20
Computing paths (not just distance)

• Maintain for each node v a predecessor node p(v)
• p(v) is initialized to be null
• Whenever an edge (u,v) is relaxed such that d(v)
  improves, then p(v) can be set to be u
• Paths can be generated from this data structure

21
All pairs algorithms using single source
algorithms

• Call a single source algorithm from each vertex s
  in V
• O(V X) where X is the running time of the given
  algorithm
• Dijkstra linear array O(V3)
• Dijkstra binary heap O(VE log V)
• Dijkstra Fibonacci heap O(V2 log V VE)
• Bellman-Ford O(V2 E) (negative weight edges)

22
Two adjacency matrix based algorithms

• Matrix-multiplication based algorithm
• Let Lm(i,j) denote the length of the shortest
  path from node i to node j using at most m edges
• What is our desired result in terms of Lm(i,j)?
• What is a recurrence relation for Lm(i,j)?
• Floyd-Warshall algorithm
• Let Lk(i,j) denote the length of the shortest
  path from node i to node j using only nodes
  within 1, , k as internal nodes.
• What is our desired result in terms of Lk(i,j)?
• What is a recurrence relation for Lk(i,j)?

23
Conceptual pictures
Shortest path using at most 2m edges
Try all possible nodes k
i
k
j
Shortest path using at most m edges
Shortest path using at most m edges
Shortest path using nodes 1 through k
Shortest path using nodes 1 through k-1
i
k
j
Shortest path using nodes 1 through k-1
OR
Shortest path using nodes 1 through k-1
24
Running Times

• Matrix-multiplication based algorithm
• O(V3 log V)
• log V executions of matrix-matrix
  multiplication
• Not quite matrix-matrix multiplication but same
  running time
• Floyd-Warshall algorithm
• O(V3)
• V iterations of an O(V2) update loop
• The constant is very small, so this is a fast
  O(V3)

25
Johnsons Algorithm

• Key ideas
• Reweight edge weights to eliminate negative
  weight edges AND preserve shortest paths
• Use Bellman-Ford and Dijkstras algorithms as
  subroutines
• Running time O(V2 log V VE)
• Better than earlier algorithms for sparse graphs

26
Reweighting

• Original edge weight is w(u,v)
• New edge weight
• w(u,v) w(u,v) h(u) h(v)
• h(v) is a function mapping vertices to real
  numbers
• Key observation
• Let p be any path from node u to node v
• w(p) w(p) h(u) h(v)

27
Computing vertex weights h(v)

• Create a new graph G (V, E) by
• adding a new vertex s to V
• adding edges (s,v) for all v in V with w(s,v) 0
• Set h(v) to be the length of the shortest path
  from this new node s to node v
• This is well-defined if G does not contain
  negative weight cycles
• Note that h(v) h(u) w(u,v) for all (u,v) in
  E
• Thus, w(u,v) w(u,v) h(u) h(v) 0

28
Algorithm implementation

• Run Bellman-Ford on G from new node s
• If no negative weight cycle, then use h(v) values
  from Bellman-Ford
• Now compute w(u,v) for each edge (u,v) in E
• Now run Dijkstras algorithm using w
• Use each node as source node
• Modify du,v at end by adding h(v) and
  subtracting h(u) to get true path weight
• Running time
• O(VE) from one run of Bellman-Ford
• O(V2 log V VE) from V runs of Dijkstra