CS%203343:%20Analysis%20of%20Algorithms - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

CS%203343:%20Analysis%20of%20Algorithms

Description:

CS 3343: Analysis of Algorithms Lecture 22: Minimum Spanning Tree – PowerPoint PPT presentation

Number of Views:15
Avg rating:3.0/5.0
Slides: 95
Provided by: Jianh150
Learn more at: http://www.cs.utsa.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: CS%203343:%20Analysis%20of%20Algorithms


1
CS 3343 Analysis of Algorithms
  • Lecture 22 Minimum Spanning Tree

2
Outline
  • Review of last lecture
  • Prims algorithm and Kruskals algorithm for MST
    in detail

3
Graph Review
  • G (V, E)
  • V a set of vertices
  • E a set of edges
  • Degree number of edges
  • In-degree vs out-degree
  • Types of graphs
  • Directed / undirected
  • Weighted / un-weighted
  • Dense / sparse
  • Connected / disconnected

4
Graph representations
  • Adjacency matrix

A 1 2 3 4
1 0 1 1 0
2 0 0 1 0
3 0 0 0 0
4 0 0 1 0
1
2
4
3
How much storage does the adjacency matrix
require? A O(V2)
5
Graph representations
  • Adjacency list

1
2
3
3
2
4
3
3
How much storage does the adjacency list
require? A O(VE)
6
Graph representations
  • Undirected graph

1
2
4
3
2
3
1
3
1
2
4
3
7
Graph representations
  • Weighted graph

1
5
6
2
4
9
4
3
2,5
3,6
1,5
3,9
1,6
2,9
4,4
3,4
8
Analysis of time complexity
  • Convention
  • Number of vertices n, V, V
  • Number of edges m, E, E
  • O(VE) is the same as O(nm) or O(VE)

9
Tradeoffs between the two representations
V n, E m
Adj Matrix Adj List
test (u, v) ? E T(1) O(n)
Degree(u) T(n) O(n)
Memory T(n2) T(nm)
Edge insertion T(1) T(1)
Edge deletion T(1) O(n)
Graph traversal T(n2) T(nm)
Both representations are very useful and have
different properties, although adjacency lists
are probably better for most problems
10
Minimum Spanning Tree
  • Problem given a connected, undirected, weighted
    graph

11
Minimum Spanning Tree
  • Problem given a connected, undirected, weighted
    graph, find a spanning tree using edges that
    minimize the total weight

6
4
5
9
  • A spanning tree is a tree that connects all
    vertices
  • Number of edges ?
  • A spanning tree has no designated root.

14
2
10
15
3
8
12
Minimum Spanning Tree
  • MSTs satisfy the optimal substructure property
    an optimal tree is composed of optimal subtrees
  • Let T be an MST of G with an edge (u,v) in the
    middle
  • Removing (u,v) partitions T into two trees T1 and
    T2
  • w(T) w(u,v) w(T1) w(T2)
  • Claim 1 T1 is an MST of G1 (V1, E1), and T2 is
    an MST of G2 (V2, E2)
  • Proof by contradiction
  • if T1 is not optimal, we can replace T1 with a
    better spanning tree, T1
  • T1, T2 and (u, v) form a new spanning tree T
  • W(T) lt W(T). Contradiction.

13
Minimum Spanning Tree
  • MSTs satisfy the optimal substructure property
    an optimal tree is composed of optimal subtrees
  • Let T be an MST of G with an edge (u,v) in the
    middle
  • Removing (u,v) partitions T into two trees T1 and
    T2
  • w(T) w(u,v) w(T1) w(T2)
  • Claim 2 (u, v) is the lightest edge connecting
    G1 (V1, E1) and G2 (V2, E2)
  • Proof by contradiction
  • if (u, v) is not the lightest edge, we can remove
    it, and reconnect T1 and T2 with a lighter edge
    (x, y)
  • T1, T2 and (x, y) form a new spanning tree T
  • W(T) lt W(T). Contradiction.

T2
T1
v
u
14
Algorithms
  • Generic idea
  • Compute MSTs for sub-graphs
  • Connect two MSTs for sub-graphs with the lightest
    edge
  • Two of the most well-known algorithms
  • Prims algorithm
  • Kruskals algorithm
  • Lets first talk about the ideas behind the
    algorithms without worrying about the
    implementation and analysis

15
Prims algorithm
  • Basic idea
  • Start from an arbitrary single node
  • A MST for a single node has no edge
  • Gradually build up a single larger and larger MST

6
5
Not yet discovered
7
Fully explored nodes
Discovered but not fully explored nodes
16
Prims algorithm
  • Basic idea
  • Start from an arbitrary single node
  • A MST for a single node has no edge
  • Gradually build up a single larger and larger MST

2
6
5
9
Not yet discovered
4
7
Fully explored nodes
Discovered but not fully explored nodes
17
Prims algorithm
  • Basic idea
  • Start from an arbitrary single node
  • A MST for a single node has no edge
  • Gradually build up a single larger and larger MST

2
6
5
9
4
7
18
Prims algorithm in words
  • Randomly pick a vertex as the initial tree T
  • Gradually expand into a MST
  • For each vertex that is not in T but directly
    connected to some nodes in T
  • Compute its minimum distance to any vertex in T
  • Select the vertex that is closest to T
  • Add it to T

19
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
20
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
21
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
22
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
23
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
24
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
25
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
26
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
27
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
28
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
29
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
30
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
31
Pseudocode for Prims algorithm
  • Given G (V, E). Output a MST T.
  • Randomly select a vertex v
  • S v A V \ S T .
  • While (A is not empty)
  • find a vertex u ? A that connects to vertex v ?
    S such that w(u, v) w(x, y), for any x ? A and
    y ? S
  • S S U u A A \ u T T U (u, v).
  • End
  • Return T

32
Time complexity
  • Given G (V, E). Output a MST T.
  • Randomly select a vertex v
  • S v A V \ S T .
  • While (A is not empty)
  • find a vertex u ? A that connects to vertex v ?
    S such that w(u, v) w(x, y), for any x ? A and
    y ? S
  • S S U u A A \ u T T U (u, v).
  • End
  • Return T

n vertices
  • Time complexity n (time spent on purple line
    per vertex)
  • Naïve test all edges gt T(n m)
  • Improve keep the list of candidates in an array
    gt T(n2)
  • Better with priority queue gt T(m log n)

33
Idea 1 naive
  • find a vertex u ? A that connects to vertex v ?
    S such that w(u, v) w(x, y), for any x ? A and
    y ? S
  • min_weight infinity.
  • For each edge (x, y) ? E
  • if x ? A, y ? S, and w(x, y) lt min_weight
  • u x v y min_weight w(x, y)

time spent per vertex T(m) Total time
complexity T(nm)
34
Idea 2 distance array
  • // For each vertex v, dv is the min distance
    from v to any node already in S
  • // pv is the parent node of v in the spanning
    tree
  • For each v ? V
  • dv infinity pv null
  • Randomly select a v, dv 1 // dv1 just to
    ensure proper start
  • S A V T .
  • While (A is not empty)
  • Search d to find the smallest du gt 0.
  • S S U u A A \ u T T U (u, pu)
  • du 0.
  • For each v in adju
  • if dv gt w(u, v)
  • dv w(u, v)
  • pv u
  • End

35
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 8 1 8 8 8 8 8
p ?/ / / / / / / /
36
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 8 0 8 8 8 8 8
p ?/ / / / / / / /
37
a
6
12
9
5
b
f
g
Discovered
7
14
15
8
c
e
h
10
3
Explored
d
Not discovered
a b c d e f g h
d ?8 14 0 3 8 8 8 8
p ?/ c / c / / / /
38
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 14 0 0 8 8 8 8
p ?/ c / c / / / /
39
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 8 0 0 10 8 8 8
p ?/ d / c d / / /
40
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 0 0 0 10 8 8 8
p ?/ d / c d / / /
41
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?6 0 0 0 10 5 8 8
p ?b d / c d b / /
42
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?6 0 0 0 10 0 8 8
p ?b d / c d b / /
43
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?6 0 0 0 7 0 9 8
p ?b d / c f b f /
44
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 7 0 9 8
p ?b d / c f b f /
45
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 9 8
p ?b d / c f b f /
46
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 9 15
p ?b d / c f b f e
47
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 0 15
p ?b d / c f b f e
48
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 0 0
p ?b d / c f b f e
49
Time complexity?
  • // For each vertex v, dv is the min distance
    from v to any node already in S
  • // pv is the parent node of v in the spanning
    tree
  • For each v ? V
  • dv infinity pv null
  • Randomly select a v, dv 1 // dv1 just to
    ensure proper start
  • S . T . A V.
  • While (A is not empty)
  • Search d to find the smallest du gt 0.
  • S S U u A A \ u T T U (u, pu)
  • du 0.
  • For each v in adju
  • if dv gt w(u, v)
  • dv w(u, v)
  • pv u
  • End

n vertices
T(n) per vertex
O(n) per vertex if using adj list ?(n) per vertex
if using adj matrix
Overall complexity T (n2)
50
Idea 3 priority queue (min-heap)
  • Observation
  • In idea 2, we need to search the distance array n
    times, each time we only look for the minimum
    element.
  • Distance array size n
  • n x n n2
  • Can we do better?
  • Priority queue (heap) enables fast retrieval of
    min (or max) elements
  • log (n) for most operations

51
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
?8 8 8 8 8 8 8 8
52
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ChangeKey
c b a d e f g h
?0 8 8 8 8 8 8 8
53
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExctractMin
h b a d e f g
8 8 8 8 8 8 8
54
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ChangeKey
d b a h e f g
3 14 8 8 8 8 8
55
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExctractMin
b g a h e f
?14 8 8 8 8 8
56
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
b e a h g f
?8 10 8 8 8 8
57
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
e f a h g
10 8 8 8 8
58
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
f e a h g
5 10 6 8 8
59
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
a e g h
6 10 8 8
60
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
a e g h
6 7 9 8
61
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
e h g
7 8 9
62
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
g h
9 8
63
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
g h
9 15
64
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
h
15
65
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
66
Complete Prims Algorithm
  • MST-Prim(G, w, r)
  • Q VG
  • for each u ? Q
  • keyu ?
  • pu null
  • keyr 0
  • while (Q not empty)
  • u ExtractMin(Q)
  • for each v ? Adju
  • if (v ? Q and w(u,v) lt keyv)
  • pv u
  • ChangeKey(v, w(u,v))

Overall running time T(m log n)
n vertices
T(n) times
T(n2) times?
T(m) times (with adj list)
How often is ExtractMin() called? How often is
ChangeKey() called?
67
Kruskals algorithm in words
  • Procedure
  • Sort all edges into non-decreasing order
  • Initially each node is in its own tree
  • For each edge in the sorted list
  • If the edge connects two separate trees, then
  • join the two trees together with that edge

68
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
69
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
70
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
71
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
72
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
73
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
74
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
75
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
76
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
77
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
78
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
79
Time complexity
  • Depend on implementation
  • Pseudocode
  • sort all edges according to weights
  • T . tree(v) v for all v.
  • for each edge (u, v)
  • if tree(u) ! tree(v)
  • T T U (u, v)
  • union (tree(u), tree(v))

T(m log m) T(m log n)
m edges
Overall time complexity m log n m t n
u Naïve T(nm) Better implementation T(m log n)
80
Disjoint Set Union
  • Use a linked list to represent tree elements,
    with pointers back to root
  • treeu ! treev O(1)
  • Union(treeu,treev) Copy elements of A into
    set B by adjusting elements of A to point to B
  • Each union may take O(n) time
  • Precisely n-1 unions
  • O(n2)?

81
Disjoint Set Union
  • Better strategy and analysis
  • Always copy smaller list into larger list
  • Size of combined list is at least twice the size
    of smaller list
  • Each vertex copied at most logn times before a
    single tree emerges
  • Total number of copy operations for n vertex is
    therefore O(n log n)
  • Overall time complexity m log n m t n u
  • t time for finding tree root is constant
  • u time for n union operations is at most nlog
    (n)
  • m log n m t n u T (m log n m n log n)
    T (m log n)
  • Conclusion
  • Kruskals algorithm runs in T(m log n) for both
    dense and sparse graph

82
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
83
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
84
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
85
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
86
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
g
f
e
h
87
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
88
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
89
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
90
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
91
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
92
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
h
f
g
a
93
Disjoint Set Union
  • Better strategy and analysis
  • Always copy smaller list into larger list
  • Size of combined list is at least twice the size
    of smaller list
  • Each vertex copied at most logn times before a
    single tree emerges
  • Total number of copy operations for n vertex is
    therefore O(n log n)
  • Overall time complexity m log n m t n u
  • t time for finding tree root is constant
  • u time for n union operations is at most nlog
    (n)
  • m log n m t n u T (m log n m n log n)
    T (m log n)
  • Conclusion
  • Kruskals algorithm runs in T(m log n) for both
    dense and sparse graph
  • How about using counting sort?
  • T(m n log n)

94
Summary
  • Kruskals algorithm
  • T(m log n)
  • Possibly T(m n log n) with counting sort
  • Prims algorithm
  • With priority queue T(m log n)
  • Assume graph represented by adj list
  • With distance array T(n2)
  • Adj list or adj matrix
  • For sparse graphs priority queue wins
  • For dense graphs distance array may be better
About PowerShow.com