# CS%203343:%20Analysis%20of%20Algorithms - PowerPoint PPT Presentation

View by Category
Title:

## CS%203343:%20Analysis%20of%20Algorithms

Description:

### CS 3343: Analysis of Algorithms Lecture 22: Minimum Spanning Tree – PowerPoint PPT presentation

Number of Views:15
Avg rating:3.0/5.0
Slides: 95
Provided by: Jianh150
Category:
Tags:
Transcript and Presenter's Notes

Title: CS%203343:%20Analysis%20of%20Algorithms

1
CS 3343 Analysis of Algorithms
• Lecture 22 Minimum Spanning Tree

2
Outline
• Review of last lecture
• Prims algorithm and Kruskals algorithm for MST
in detail

3
Graph Review
• G (V, E)
• V a set of vertices
• E a set of edges
• Degree number of edges
• In-degree vs out-degree
• Types of graphs
• Directed / undirected
• Weighted / un-weighted
• Dense / sparse
• Connected / disconnected

4
Graph representations

A 1 2 3 4
1 0 1 1 0
2 0 0 1 0
3 0 0 0 0
4 0 0 1 0
1
2
4
3
How much storage does the adjacency matrix
require? A O(V2)
5
Graph representations

1
2
3
3
2
4
3
3
How much storage does the adjacency list
require? A O(VE)
6
Graph representations
• Undirected graph

1
2
4
3
2
3
1
3
1
2
4
3
7
Graph representations
• Weighted graph

1
5
6
2
4
9
4
3
2,5
3,6
1,5
3,9
1,6
2,9
4,4
3,4
8
Analysis of time complexity
• Convention
• Number of vertices n, V, V
• Number of edges m, E, E
• O(VE) is the same as O(nm) or O(VE)

9
V n, E m
test (u, v) ? E T(1) O(n)
Degree(u) T(n) O(n)
Memory T(n2) T(nm)
Edge insertion T(1) T(1)
Edge deletion T(1) O(n)
Graph traversal T(n2) T(nm)
Both representations are very useful and have
are probably better for most problems
10
Minimum Spanning Tree
• Problem given a connected, undirected, weighted
graph

11
Minimum Spanning Tree
• Problem given a connected, undirected, weighted
graph, find a spanning tree using edges that
minimize the total weight

6
4
5
9
• A spanning tree is a tree that connects all
vertices
• Number of edges ?
• A spanning tree has no designated root.

14
2
10
15
3
8
12
Minimum Spanning Tree
• MSTs satisfy the optimal substructure property
an optimal tree is composed of optimal subtrees
• Let T be an MST of G with an edge (u,v) in the
middle
• Removing (u,v) partitions T into two trees T1 and
T2
• w(T) w(u,v) w(T1) w(T2)
• Claim 1 T1 is an MST of G1 (V1, E1), and T2 is
an MST of G2 (V2, E2)
• if T1 is not optimal, we can replace T1 with a
better spanning tree, T1
• T1, T2 and (u, v) form a new spanning tree T

13
Minimum Spanning Tree
• MSTs satisfy the optimal substructure property
an optimal tree is composed of optimal subtrees
• Let T be an MST of G with an edge (u,v) in the
middle
• Removing (u,v) partitions T into two trees T1 and
T2
• w(T) w(u,v) w(T1) w(T2)
• Claim 2 (u, v) is the lightest edge connecting
G1 (V1, E1) and G2 (V2, E2)
• if (u, v) is not the lightest edge, we can remove
it, and reconnect T1 and T2 with a lighter edge
(x, y)
• T1, T2 and (x, y) form a new spanning tree T

T2
T1
v
u
14
Algorithms
• Generic idea
• Compute MSTs for sub-graphs
• Connect two MSTs for sub-graphs with the lightest
edge
• Two of the most well-known algorithms
• Prims algorithm
• Kruskals algorithm
• Lets first talk about the ideas behind the
implementation and analysis

15
Prims algorithm
• Basic idea
• Start from an arbitrary single node
• A MST for a single node has no edge
• Gradually build up a single larger and larger MST

6
5
Not yet discovered
7
Fully explored nodes
Discovered but not fully explored nodes
16
Prims algorithm
• Basic idea
• Start from an arbitrary single node
• A MST for a single node has no edge
• Gradually build up a single larger and larger MST

2
6
5
9
Not yet discovered
4
7
Fully explored nodes
Discovered but not fully explored nodes
17
Prims algorithm
• Basic idea
• Start from an arbitrary single node
• A MST for a single node has no edge
• Gradually build up a single larger and larger MST

2
6
5
9
4
7
18
Prims algorithm in words
• Randomly pick a vertex as the initial tree T
• Gradually expand into a MST
• For each vertex that is not in T but directly
connected to some nodes in T
• Compute its minimum distance to any vertex in T
• Select the vertex that is closest to T

19
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
20
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
21
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
22
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
23
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
24
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
25
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
26
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
27
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
28
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
29
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
30
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
31
Pseudocode for Prims algorithm
• Given G (V, E). Output a MST T.
• Randomly select a vertex v
• S v A V \ S T .
• While (A is not empty)
• find a vertex u ? A that connects to vertex v ?
S such that w(u, v) w(x, y), for any x ? A and
y ? S
• S S U u A A \ u T T U (u, v).
• End
• Return T

32
Time complexity
• Given G (V, E). Output a MST T.
• Randomly select a vertex v
• S v A V \ S T .
• While (A is not empty)
• find a vertex u ? A that connects to vertex v ?
S such that w(u, v) w(x, y), for any x ? A and
y ? S
• S S U u A A \ u T T U (u, v).
• End
• Return T

n vertices
• Time complexity n (time spent on purple line
per vertex)
• Naïve test all edges gt T(n m)
• Improve keep the list of candidates in an array
gt T(n2)
• Better with priority queue gt T(m log n)

33
Idea 1 naive
• find a vertex u ? A that connects to vertex v ?
S such that w(u, v) w(x, y), for any x ? A and
y ? S
• min_weight infinity.
• For each edge (x, y) ? E
• if x ? A, y ? S, and w(x, y) lt min_weight
• u x v y min_weight w(x, y)

time spent per vertex T(m) Total time
complexity T(nm)
34
Idea 2 distance array
• // For each vertex v, dv is the min distance
from v to any node already in S
• // pv is the parent node of v in the spanning
tree
• For each v ? V
• dv infinity pv null
• Randomly select a v, dv 1 // dv1 just to
ensure proper start
• S A V T .
• While (A is not empty)
• Search d to find the smallest du gt 0.
• S S U u A A \ u T T U (u, pu)
• du 0.
• For each v in adju
• if dv gt w(u, v)
• dv w(u, v)
• pv u
• End

35
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 8 1 8 8 8 8 8
p ?/ / / / / / / /
36
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 8 0 8 8 8 8 8
p ?/ / / / / / / /
37
a
6
12
9
5
b
f
g
Discovered
7
14
15
8
c
e
h
10
3
Explored
d
Not discovered
a b c d e f g h
d ?8 14 0 3 8 8 8 8
p ?/ c / c / / / /
38
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 14 0 0 8 8 8 8
p ?/ c / c / / / /
39
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 8 0 0 10 8 8 8
p ?/ d / c d / / /
40
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?8 0 0 0 10 8 8 8
p ?/ d / c d / / /
41
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?6 0 0 0 10 5 8 8
p ?b d / c d b / /
42
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?6 0 0 0 10 0 8 8
p ?b d / c d b / /
43
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?6 0 0 0 7 0 9 8
p ?b d / c f b f /
44
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 7 0 9 8
p ?b d / c f b f /
45
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 9 8
p ?b d / c f b f /
46
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 9 15
p ?b d / c f b f e
47
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 0 15
p ?b d / c f b f e
48
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
d ?0 0 0 0 0 0 0 0
p ?b d / c f b f e
49
Time complexity?
• // For each vertex v, dv is the min distance
from v to any node already in S
• // pv is the parent node of v in the spanning
tree
• For each v ? V
• dv infinity pv null
• Randomly select a v, dv 1 // dv1 just to
ensure proper start
• S . T . A V.
• While (A is not empty)
• Search d to find the smallest du gt 0.
• S S U u A A \ u T T U (u, pu)
• du 0.
• For each v in adju
• if dv gt w(u, v)
• dv w(u, v)
• pv u
• End

n vertices
T(n) per vertex
O(n) per vertex if using adj list ?(n) per vertex
Overall complexity T (n2)
50
Idea 3 priority queue (min-heap)
• Observation
• In idea 2, we need to search the distance array n
times, each time we only look for the minimum
element.
• Distance array size n
• n x n n2
• Can we do better?
• Priority queue (heap) enables fast retrieval of
min (or max) elements
• log (n) for most operations

51
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a b c d e f g h
?8 8 8 8 8 8 8 8
52
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ChangeKey
c b a d e f g h
?0 8 8 8 8 8 8 8
53
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExctractMin
h b a d e f g
8 8 8 8 8 8 8
54
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ChangeKey
d b a h e f g
3 14 8 8 8 8 8
55
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExctractMin
b g a h e f
?14 8 8 8 8 8
56
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
b e a h g f
?8 10 8 8 8 8
57
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
e f a h g
10 8 8 8 8
58
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
f e a h g
5 10 6 8 8
59
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
a e g h
6 10 8 8
60
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
a e g h
6 7 9 8
61
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
e h g
7 8 9
62
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
g h
9 8
63
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
Changekey
g h
9 15
64
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
ExtractMin
h
15
65
Example
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
66
Complete Prims Algorithm
• MST-Prim(G, w, r)
• Q VG
• for each u ? Q
• keyu ?
• pu null
• keyr 0
• while (Q not empty)
• u ExtractMin(Q)
• for each v ? Adju
• if (v ? Q and w(u,v) lt keyv)
• pv u
• ChangeKey(v, w(u,v))

Overall running time T(m log n)
n vertices
T(n) times
T(n2) times?
How often is ExtractMin() called? How often is
ChangeKey() called?
67
Kruskals algorithm in words
• Procedure
• Sort all edges into non-decreasing order
• Initially each node is in its own tree
• For each edge in the sorted list
• If the edge connects two separate trees, then
• join the two trees together with that edge

68
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
69
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
70
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
71
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
72
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
73
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
74
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
75
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
76
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
77
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
78
Example
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
79
Time complexity
• Depend on implementation
• Pseudocode
• sort all edges according to weights
• T . tree(v) v for all v.
• for each edge (u, v)
• if tree(u) ! tree(v)
• T T U (u, v)
• union (tree(u), tree(v))

T(m log m) T(m log n)
m edges
Overall time complexity m log n m t n
u Naïve T(nm) Better implementation T(m log n)
80
Disjoint Set Union
• Use a linked list to represent tree elements,
with pointers back to root
• treeu ! treev O(1)
• Union(treeu,treev) Copy elements of A into
set B by adjusting elements of A to point to B
• Each union may take O(n) time
• Precisely n-1 unions
• O(n2)?

81
Disjoint Set Union
• Better strategy and analysis
• Always copy smaller list into larger list
• Size of combined list is at least twice the size
of smaller list
• Each vertex copied at most logn times before a
single tree emerges
• Total number of copy operations for n vertex is
therefore O(n log n)
• Overall time complexity m log n m t n u
• t time for finding tree root is constant
• u time for n union operations is at most nlog
(n)
• m log n m t n u T (m log n m n log n)
T (m log n)
• Conclusion
• Kruskals algorithm runs in T(m log n) for both
dense and sparse graph

82
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
83
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
84
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
85
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
e
f
g
h
86
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
a
b
c
d
g
f
e
h
87
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
88
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
89
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
90
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
91
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
a
f
g
h
92
Example with disjoint set union
c-d 3 b-f 5 b-a 6 f-e 7 b-d 8 f-g
9 d-e 10 a-f 12 b-c 14 e-h 15
a
6
12
9
5
b
f
g
7
14
15
8
c
e
h
10
3
d
e
b
c
d
h
f
g
a
93
Disjoint Set Union
• Better strategy and analysis
• Always copy smaller list into larger list
• Size of combined list is at least twice the size
of smaller list
• Each vertex copied at most logn times before a
single tree emerges
• Total number of copy operations for n vertex is
therefore O(n log n)
• Overall time complexity m log n m t n u
• t time for finding tree root is constant
• u time for n union operations is at most nlog
(n)
• m log n m t n u T (m log n m n log n)
T (m log n)
• Conclusion
• Kruskals algorithm runs in T(m log n) for both
dense and sparse graph
• How about using counting sort?
• T(m n log n)

94
Summary
• Kruskals algorithm
• T(m log n)
• Possibly T(m n log n) with counting sort
• Prims algorithm
• With priority queue T(m log n)
• Assume graph represented by adj list
• With distance array T(n2)