# In The Name Of God Algorithms Design Greedy Algorithms - PowerPoint PPT Presentation

PPT – In The Name Of God Algorithms Design Greedy Algorithms PowerPoint presentation | free to download - id: 691296-ZDBmM

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## In The Name Of God Algorithms Design Greedy Algorithms

Description:

### Title: Greedy Subject: Algorithms Design Author: bijani Last modified by: Zenith Created Date: 9/17/2007 1:03:40 PM Document presentation format: On-screen Show (4:3) – PowerPoint PPT presentation

Number of Views:29
Avg rating:3.0/5.0
Slides: 56
Provided by: bijani
Category:
Tags:
Transcript and Presenter's Notes

Title: In The Name Of God Algorithms Design Greedy Algorithms

1
In The Name Of GodAlgorithms DesignGreedy
Algorithms
• Dr. Shahriar Bijani
• Shahed University
• Feb. 2014

2
Slides References
• Kleinberg and Tardos, Algorithm Design, CSE 5311,
M Kumar, Spring 2007.
• Chapter 4, Computer Algorithms, Horowitz ,,
2001.
• Chapter 16, Introduction to Algorithms, CLRS ,
2001.

3
The Greedy Principle
• The problem We are required to find a feasible
solution that either maximizes or minimizes a
given objective solution.
• It is easy to determine a feasible solution but
not necessarily an optimal solution.
• The greedy method solves this problem in stages,
at each stage, a decision is made considering
inputs in an order determined by the selection
procedure which may be based on an optimization
measure.

4
Optimization problems
• An optimization problem is one in which you want
to find, not just a solution, but the best
solution
• A greedy algorithm sometimes works well for
optimization problems
• A greedy algorithm works in phases. At each
phase
• You take the best you can get right now, without
regard for future consequences
• You hope that by choosing a local optimum at each
step, you will end up at a global optimum

5
Example Counting money
• Suppose you want to count out a certain amount of
money, using the fewest possible bills and coins
• A greedy algorithm would do this would beAt
each step, take the largest possible bill or coin
that does not overshoot
• Example To make 6.39, you can choose
• a 5 bill
• a 1 bill, to make 6
• a 25 coin, to make 6.25
• A 10 coin, to make 6.35
• four 1 coins, to make 6.39
• For US money, the greedy algorithm always gives
the optimum solution

6
A failure of the greedy algorithm
• In some (fictional) monetary system, Rial come
in 1 Rial, 7 Rial, and 10 Rial coins
• Using a greedy algorithm to count out 15 Rials,
you would get
• A 10 Rial piece
• Five 1 Rial pieces, for a total of 15 Rials
• This requires six coins
• A better solution would be to use two 7 Rial
pieces and one 1 Rial piece
• This only requires three coins
• This greedy algorithm results in a solution, but
not in an optimal solution

7
A scheduling problem
• You have to run nine jobs, with running times of
3, 5, 6, 10, 11, 14, 15, 18, and 20 minutes
• You have three processors on which you can run
these jobs
• You decide to do the longest-running jobs first,
on whatever processor is available

P1 P2 P3
• Time to completion 18 11 6 35 minutes
• This solution isnt bad, but we might be able to
do better

8
Another approach
• What would be the result if you ran the shortest
job first?
• Again, the running times are 3, 5, 6, 10, 11, 14,
15, 18, and 20 minutes

P1 P2 P3
• That wasnt such a good idea time to completion
is now 6 14 20 40 minutes
• Note, however, that the greedy algorithm itself
is fast
• All we had to do at each stage was pick the
minimum or maximum

9
An optimum solution
• Better solutions do exist
• This solution is clearly optimal (why?)
• Clearly, there are other optimal solutions (why?)
• How do we find such a solution?
• One way Try all possible assignments of jobs to
processors
• Unfortunately, this approach can take exponential
time

10
Huffman encoding
• The Huffman encoding algorithm is a greedy
algorithm
• You always pick the two smallest numbers to
combine
• Average bits/char0.222 0.123 0.242
0.064 0.272 0.094 2.42
• The Huffman algorithm finds an optimal solution

A00B100C01D1010E11F1011
22 12 24 6 27 9 A B C D E
F
11
Procedure Huffman_Encoding(S,f) Input S (a
string of characters) and f (an array of
frequencies). Output T (the Huffman tree for
S) 1. insert all characters into a heap H
according to their frequencies 2. while H
is not empty do 3. if H contains only one
character x then 4. x ? root (T) 5. else
6. z ? ALLOCATE_NODE() 7. x ? leftT,z ?
EXTRACT_MIN(H) 8. y ? rightT,z ?
EXTRACT_MIN(H) 9. fz ? fx fy 10. INSERT(H,z)

12
Complexity of Huffman algorithm Building a heap
in step 1 takes O(n) time Insertions (steps 7 and
8) and deletions (step 10) on H take O (log
n) time each Therefore Steps 2 through 10 take
O(n logn) time Thus the overall complexity of
the algorithm is O( n logn ).
13
Knapsack Problem
• Assume a thief is robbing a store with various
valued and weighted objects. Her bag can hold
only W pounds.
• The 0-1 knapsack problem must take all or
nothing
• The fractional knapsack problem can take all or
part of the item

14
Where Greedy Fails
50
50
50
50
50
20
20 --- 30
30
30
30
20
20
0-1 knapsack
20
10
10
10
10
60
220
160
180
100
120
240
15
Fractional Knapsack Problem
• Given n items of sizes w1, w2, , wn and values
p1, p2, , pn and size of knapsack of C, the
problem is to find x1, x2, , xn ?? that maximize
• subject to

16
Solution to Fractional Knapsack Problem
• Consider yi pi / wi
• What is yi?
• What is the solution?
• fractional problem has greedy-choice property,
0-1 does not.

17
Critical Path Method (CPM)
• Activity On Edge (AOE) Networks
• Events v0,v1,

V6
V1
a3 1
a6 9
a9 2
a0 6
V0
V4
V8
finish
start
a1 4
a10 4
a4 1
a7 7
V7
V2
• Some definition
• Predecessor
• Successor
• Immediate predecessor
• Immediate successor

a2 5
a8 4
a5 2
V3
V5
18
CPM critical path
• A critical path is a path that has the longest
length. (v0, v1, v4, v7, v8)

V6
V1
a3 1
a6 9
a9 2
a0 6
V0
V4
V8
start
finish
a1 4
a10 4
a4 1
a7 7
V7
V2
a2 5
a8 4
6 1 7 4 18 (Max)
a5 2
V3
V5
19
The earliest time
• The earliest time of an activity, ai, can occur
is the length of the longest path from the start
vertex v0 to ais start vertex. (Ex the
earliest time of activity a7 can occur is 7.)
• We denote this time as early(i) for activity
ai.? early(6) early(7) 7.

V6
V1
a3 1
a6 9
a9 2
a0 6
7/?
6/?
16/?
0/?
V0
V4
V8
finish
start
a1 4
14/?
7/?
4/?
18
a10 4
a4 1
a7 7
0/?
V7
V2
0/?
a2 5
a8 4
7/?
a5 2
V3
V5
5/?
20
The latest time
• The latest time, late(i), of activity, ai, is
defined to be the latest time the activity may
• Ex early(5) 5 late(5) 8 early(7) 7
late(7) 7

V6
V1
a3 1
a6 9
a9 2
a0 6
7/7
6/6
16/16
0/0
V0
V4
V8
finish
start
a1 4
14/14
7/7
4/5
a10 4
a4 1
a7 7
0/1
V7
V2
0/3
a2 5
a8 4
7/10
late(5) 18 4 4 - 2 8 late(7) 18 4 7
7
a5 2
V3
V5
5/8
21
Critical activity
• A critical activity is an activity for which
early(i) late(i).
• The difference between late(i) and early(i) is a
measure of how critical an activity is.

CalculationofEarliest Times
FindingCritical path(s)
To solveAOE Problem
CalculationofLatest Times
22
Calculation of Earliest Times
• Let activity ai is represented by edge (u, v).
• early (i) earliest u
• late (i) latest v duration of activity ai
• We compute the times in two stagesa forward
stage and a backward stage.
• The forward stage
• Step 1 earliest 0 0
• Step 2 earliest j max earliest i
duration of (i, j) i is in P(j)
• P(j) is the set of immediate predecessors of j.

23
Calculation of Latest Times
• The backward stage
• Step 1 latestn-1 earliestn-1
• Step 2 latest j min latest i - duration
of (j, i) i is in S(j)
• S(j) is the set of vertices adjacent from vertex
j.
• latest8 earliest8 18
• latest6 minearliest8 - 2 16
• latest7 minearliest8 - 4 14
• latest4 minearliest6 9 earliest7
7 7
• latest1 minearliest4 - 1 6
• latest2 minearliest4 - 1 6
• latest5 minearliest7 - 4 10
• latest3 minearliest5 - 2 8
• latest0 minearliest1 6 earliest2
4 earliest3 5 0

24
Graph with non-critical activities deleted
V6
V1
Activity Early Late L - E Critical
a0 0 0 0 Yes
a1 0 2 2 No
a2 0 3 3 No
a3 6 6 0 Yes
a4 4 6 2 No
a5 5 8 3 No
a6 7 7 0 Yes
a7 7 7 0 Yes
a8 7 10 3 No
a9 16 16 0 Yes
a10 14 14 0 Yes
a9
a0
a3
a6
V8
V0
V4
a1
V2
V7
finish
start
a4
a7
a10
a2
V3
V5
a8
a5
V6
a6
a9
V1
a0
a3
V0
V4
V8
start
finish
V7
a7
a10
25
CPM for the longest path problem
• The longest path(critical path) problem can be
solved by the critical path method(CPM)
• Step 1Find a topological ordering.
• Step 2 Find the critical path.

26
Traveling salesman
• A salesman must visit every city (starting from
city A), and wants to cover the least possible
distance
• He can revisit a city (and reuse a road) if
necessary
• He does this by using a greedy algorithm He goes
to the next nearest city from wherever he is
• From A he goes to B
• From B he goes to D
• This is not going to result in a shortest path!
• The best result he can get now will be ABDBCE, at
a cost of 16
• An actual least-cost path from A is ADBCE, at a
cost of 14

27
Analysis
• A greedy algorithm typically makes
(approximately) n choices for a problem of size n
• (The first or last choice may be forced)
• Hence the expected running time isO(n
O(choice(n))), where choice(n) is making a choice
among n objects
• Counting Must find largest useable coin from
among k sizes of coin (k is a constant), an
O(k)O(1) operation
• Therefore, coin counting is (n)
• Huffman Must sort n values before making n
choices
• Therefore, Huffman is O(n log n) O(n) O(n log
n)

28
Some Other greedy algorithms
• Dijkstras algorithm for finding the shortest
path in a graph (single source)
• Always takes the shortest edge connecting a known
node to an unknown node
• Prims algorithm for finding a minimum-cost
spanning tree
• Always takes the lowest-cost edge between nodes
in the spanning tree and nodes not yet in the
spanning tree
• Kruskals algorithm for finding a minimum-cost
spanning tree
• Always tries the lowest-cost remaining edge

29
Single Source Shortest Path
• In a shortest-paths problem, we are given a
weighted, directed graph G (V,E), with weights
assigned to each edge in the graph. The weight of
the path p (v0, v1, v2, , vk) is the sum of
the weights of its edges
• v0 ? v1 ? v2 . . . ? vk-1? vk
• The shortest-path from u to v is given by
d(u,v) min weight (p) if there are one or
more paths from u to v
• ? otherwise

30
Dijkstra Single-Source Shortest Paths
Given G (V,E), find the shortest path from a
given vertex u ? V to every vertex v ? V ( u
?v). For each vertex v ? V in the weighted
directed graph, dv represents the distance from
u to v. Initially, dv 0 when u v. dv
? if (u,v) is not an edge dv
weight of edge (u,v) if (u,v) exists. Dijkstra's
Algorithm At every step of the algorithm, we
compute, dy min dy, dx w(x,y),
where x,y ? V. Dijkstra's algorithm is based on
the greedy principle because at every step we
pick the path of least weight.
31
(No Transcript)
32

Step Vertex to be marked Distance to vertex Distance to vertex Distance to vertex Distance to vertex Distance to vertex Distance to vertex Distance to vertex Distance to vertex Distance to vertex Unmarked vertices
Step Vertex to be marked u a b c d e f g h Unmarked vertices
0 u 0 1 5 ? 9 ? ? ? ? a,b,c,d,e,f,g,h
1 a 0 1 5 3 9 ? ? ? ? b,c,d,e,f,g,h
2 c 0 1 5 3 7 ? 12 ? ? b,d,e,f,g,h
3 b 0 1 5 3 7 8 12 ? ? d,e,f,g,h
4 d 0 1 5 3 7 8 12 11 ? e,f,g,h
5 e 0 1 5 3 7 8 12 11 9 f,g,h
6 h 0 1 5 3 7 8 12 11 9 g,h
7 g 0 1 5 3 7 8 12 11 9 h
8 f 0 1 5 3 7 8 12 11 9 --
33
Dijkstra's Single-source shortest path
• Procedure Dijkstra's Single-source shortest
path_G(V,E,u)
• Input G (V,E), the weighted directed graph and
v the source vertex
• Output for each vertex, v, dv is the length of
the shortest path from u to v.
• mark vertex u
• du ? 0
• for each unmarked vertex v ? V do
• if edge (u,v) exists d v ? weight (u,v)
• else dv ? ?
• while there exists an unmarked vertex do
• let v be an unmarked vertex such that dv is
minimal
• mark vertex v
• for all edges (v,x) such that x is unmarked do
• if dx gt dv weightv,x then
• dx ? dv weightv,x

34
Dijkstra's Single-source shortest path
• Complexity of Dijkstra's algorithm
• Steps 1 and 2 take ? (1) time
• Steps 3 to 5 take O(?V?) time
• The vertices are arranged in a heap in order of
their paths from u
• Updating the length of a path takes O(log V)
time.
• There are ?V? iterations, and at most ?E?
• Therefore the algorithm takes O((?E??V?) log
?V?) time.

35
Minimum Spanning Tree (MST)
• Spanning tree of a connected graph G a connected
acyclic subgraph of G that includes all of Gs
vertices
• Minimum spanning tree of a weighted, connected
graph G a spanning tree of G of the minimum
total weight
• MST nn-2 (n numbers of nodes)
• Example

36
Prims MST algorithm
and grow tree one vertex at a time to produce
MST through a series of expanding subtrees T1,
T2, , Tn
• On each iteration, construct Ti1 from Ti by
adding vertex not in Ti that is closest to those
already in Ti (this is a greedy step!)
• Stop when all vertices are included

37
Example

38
• Proof by induction that this construction
actually yields an MST (CLRS, Ch. 23.1).
• Efficiency
• O(n2) for weight matrix representation of graph
and array implementation of priority queue
• O(m log n) for adjacency list representation of
graph with n vertices and m edges and min-heap
implementation of the priority queue

39
Kruskals_ MST
• The algorithm maintains a collection VS of
disjoint sets of vertices
• Each set W in VS represents a connected set of
vertices forming a spanning tree
• Initially, each vertex is in a set by itself in
VS
• Edges are chosen from E in order of increasing
cost, we consider each edge (v, w) in turn v, w
? V.
• If v and w are already in the same set (say W) of
• If v and w are in distinct sets W1 and W2
(meaning v and/or w not in T) we merge W1 with
W2 and add (v, w) to T.

40
Kruskal's Algorithm
• Procedure MCST_G(V,E)
• (Kruskal's Algorithm)
• Input An undirected graph G(V,E) with a cost
function c on the edges
• Output T the minimum cost spanning tree for G
• T ? 0
• VS ?0
• for each vertex v ? V do
• VS VS ? v
• sort the edges of E in nondecreasing order of
weight
• while ?VS? gt 1 do
• choose (v,w) an edge E of lowest cost
• delete (v,w) from E
• if v and w are in different sets W1 and W2 in
VS do
• W1 W1 ? W2
• VS VS - W2
• T ? T? (v,w)
• return T

41
Kruskals_ MST
• Consider the example graph shown earlier, The
edges in nondecreasing order
• (A,D),1,(C,D),1,(C,F),2,(E,F),2,(A,F),3,
(A,B),3,
• (B,E),4,(D,E),5,(B,C),6
• EdgeActionSets in VS Spanning Tree, T
A,B,C,D,E,F0(A,D) merge
A,D, B,C, E, F (A,D) (C,D) merge
A,C,D, B, E, F (A,D), (C,D) (C,F)
mergeA,C,D,F,B,E(A,D),(C,D), (C,F)
(E,F) mergeA,C,D,E,F,B(A,D),(C,D),
(C,F),(E,F)(A,F) rejectA,C,D,E,F,B(A,D),
(C,D), (C,F), (E,F)(A,B) mergeA,B,C,D,E,F(
A,D),(A,B),(C,D), (C,F),(E,F)(B,E) reject(D,E)
reject(B,C) reject

42
Kruskals_ MST Complexity
• Steps 1 thru 4 take time O (V)
• Step 5 sorts the edges in nondecreasing order in
O (E log E ) time
• Steps 6 through 13 take O (E) time
• The total time for the algorithm is therefore
given by O (E log E)
• The edges can be maintained in a heap data
structure with the property, EPARENT(i) ? Ei
• This property can be used to sort data elements
in nonincreasing order.
• Construct a heap of the edge weights, the edge
with lowest cost is at the root During each step
of edge removal, delete the root (minimum
element) from the heap and rearrange the heap.
• The use of heap data structure reduces the time
taken because at every step we are only picking
up the minimum or root element rather than
sorting the edge weights.

43
Minimum-Cost Spanning Trees (Application)
• Consider a network of computers connected through
a positive cost the cost of sending a message on
• This network can be represented by an undirected
graph with positive costs on each edge.
• In bidirectional networks we can assume that the
cost of sending a message on a link does not
depend on the direction.
• Suppose we want to broadcast a message to all the
computers from an arbitrary computer.
• The cost of the broadcast is the sum of the costs
of links used to forward the message.

44
MST (MCST) Application
45
Activity Selection Problem
• Problem Formulation
• Given a set of n activities, S a1, a2, ...,
an that require exclusive use of a common
resource, find the largest possible set of
nonoverlapping activities (also called mutually
compatible).
• For example, scheduling the use of a classroom.
• Assume that ai needs the resource during period
si, fi), which is a half-open interval, where
• si start time of the activity, and
• fi finish time of the activity.
• Note Could have many other objectives
• Schedule room for longest time.
• Maximize income rental fees.

46
Activity Selection Problem Example
• Assume the following set S of activities that are
sorted by their finish time, find a maximum-size
mutually compatible set.

i 1 2 3 4 5 6 7 8 9
si 1 2 4 1 5 8 9 11 13
fi 3 5 7 8 9 10 11 14 16
47
i 1 2 3 4 5 6 7 8 9
si 1 2 4 1 5 8 9 11 13
fi 3 5 7 8 9 10 11 14 16
48
Solving the Activity Selection Problem
• Define Si,j ak ?? S fi ? sk lt fk ? sj
• activities that start after ai finishes and
finish before aj starts
• Activities in Si,j are compatible with
• ai and aj
• aw that finishes not later than ai and start not
earlier than aj
• Add the following imaginary activities
• a0 ?? , 0) and an1 ? , ?1)
• Hence, S S0,n1 and the range of Si,j is
0 ? i,j ? n1

49
Solving the Activity Selection Problem
• Assume that activities are sorted by
monotonically increasing finish time
• i.e., f0 ? f1 ? f2 ? ... ? fn lt fn1
• Then, Si,j ?? for i ? j.
• Proof
• Therefore, we only need to worry about Si,j where
0 ? i lt j ? n1

50
Solving the Activity Selection Problem
• Suppose that a solution to Si,j includes ak . We
have 2 sub-problems
• Si,k (start after ai finishes, finish before ak
starts)
• Sk,j (start after ak finishes, finish before aj
starts)
• The Solution to Si,j is
• (solution to Si,k ) ?? ak ? (solution to
Sk,j )
• Since ak is in neither sub-problem, and the
subproblems are disjoint,
• solution to S solution to Si,k 1
solution to Sk,j

51
Recursive Solution
• Let Ai,j optimal solution to Si,j .
• So Ai,j Ai,k ? ak ? Ak,j, assuming
• Si,j is nonempty, and
• we know ak.
• Hence,
• ci,j number of activities in a max subset of
mutually compatible activities

52
Finding the Greedy Algorithm
• Theorem Let Si,j ? ?, and let am be the activity
in Si,j with the earliest finish time fm min
fk ak ? Si,j . Then
• am is used in some maximum-size subset of
mutually compatible activities of Si,j
• Sim ?, so that choosing am leaves Sm,j as the
only nonempty subproblem.

53
Recursive Greedy Algorithm
54
Example
55
Iterative Greedy Algorithm
?(Sorting)?(n) ? (n log n)
56
Collecting coins
• A board has a certain number of coins on it
• A robot starts in the upper-left corner, and
walks to the bottom left-hand corner
• The robot can only move in two directions right
and down
• The robot collects coins as it goes
• You want to collect all the coins using the
minimum number of robots
• Example
• Do you see a greedy algorithm for doing this?
• Does the algorithm guarantee an optimal solution?
• Can you prove it?
• Can you find a counterexample?