Greedy Methods

1
Greedy Methods

• Binhai Zhu
• Computer Science Department, Montana State
  University

2
Idea

• You want to maximize a global function, which
  could be hard
• (1) You always make the best local decision in
  the hope of getting the overall best solution.
• (2) The idea is natural (and in some situation
  it is the best a human can do).

3
Idea

• You want to maximize a global function, which
  could be hard
• (1) You always make the best local decision in
  the hope of getting the overall best solution.
• (2) The idea is natural (and in some situation
  it is the best a human can do).
• (3) Of course, sometimes it might not work.

4
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. Remember that in this case a
penny and a dime is considered the same (possibly
in weight).
5
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. Remember that in this case a
penny and a dime is considered the same (possibly
in weight). Lets first look at the US system.
1
25
5
10
6
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 2.17?
1
25
5
10
7
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 2.17? 8
quarters 2.00 1 dime 0.10 1 nickel
0.05 2 pennies 0.02 So we use 12 coins!
1
25
5
10
8
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 2.17? 8
quarters 2.00 1 dime 0.10 1 nickel
0.05 2 pennies 0.02 Why this works?
1
25
5
10
9
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 2.17? 8
quarters 2.00 1 dime 0.10 1 nickel
0.05 2 pennies 0.02 Why this works? If we
make m x 50 of changes using quarters and dimes,
then the number of quarters used lt
the number of dimes used
10
1
25
5
10
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 0.15
under a new system with 3 coins?
C3
11
C1
1
5
C2
11
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 0.15
under a new system with 3 coins? Greedy 1 C1
11 4 C3 4 So we will have to use 5 coins.
C3
11
C1
1
5
C2
12
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 0.15
under a new system with 3 coins? Greedy 1 C1
11 4 C3 4 A better way 3 C2, only 3 coins!
C3
11
C1
1
5
C2
13
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 0.70
under a new system with 3 coins? Greedy 6 C1
66 4 C3 4 A better way?
C3
11
C1
1
5
C2
14
Example 1. Coin Change
Making changes for n cents using the minimum
number of coins. How to make change for 0.70
under a new system with 3 coins? Greedy 6 C1
66 4 C3 4 A better way? 5 C1 55 3 C2
15 While greedy methods do not always work (in
fact, nothing always works), they are still
useful in some situations.
C3
11
C1
1
5
C2
15
Example 2. The knapsack problem
You have a knapsack which can only contain
certain weight C of goods. With this weight
capacity constraint, you want to maximize the
values of the goods you can put in the
knapsack. G1candy, Total value1.0,
Total weight10 pounds G2chocolate, Total
value2.0, Total weight1 pounds G3ice cream,
Total value2.5, Total weight4 pounds If C4
pounds, what would you do?
16
Example 2. The knapsack problem
G1candy, Total value1.0, Total weight10
pounds G2chocolate, Total value2.0, Total
weight1 pounds G3ice cream, Total value2.5,
Total weight4 pounds If C4 pounds, what would
you do? Greedy 1 by maximum value, 4 pounds of
ice cream, profit2.5 Greedy 2 by maximum
weight, 4 pounds of candy, profit0.4 Greedy 3
by maximum unit value, 1 pound of chocolate
followed with 3 pounds of ice cream, profit3.875
17
Example 2. The knapsack problem
G1candy, Total value1.0, Total weight10
pounds G2chocolate, Total value2.0, Total
weight1 pounds G3ice cream, Total value2.5,
Total weight4 pounds In general, you have
G1,G2,,Gn, each Gi with weight wi and value
vi, and you want to maximize the profit out of
the goods you can put in the knapsack with
capacity C. How do we formulate this as a
mathematical programming problem?
18
Example 2. The knapsack problem
In general, you have G1,G2,,Gn, each Gi with
weight wi and value vi, and you want to maximize
the profit out of the goods you can put in the
knapsack with capacity C. How do we formulate
this as a mathematical programming problem? Let
fi be the fractional of Gi one would put in the
knapsack.
19
Example 2. The knapsack problem
In general, you have G1,G2,,Gn, each Gi with
weight wi and value vi, and you want to maximize
the profit out of the goods you can put in the
knapsack with capacity C. How do we formulate
this as a mathematical programming problem? Let
fi be the fractional of Gi one would put in the
knapsack. Maximize ?i1..nfivi
Subject to ?i1..nfiwiC,
0fi1,i1..n Fractional Knapsack
Problem
20
Example 2. The knapsack problem
In general, you have G1,G2,,Gn, each Gi with
weight wi and value vi, and you want to maximize
the profit out of the goods you can put in the
knapsack with capacity C. How do we formulate
this as a mathematical programming problem? Let
fi be the fractional of Gi one would put in the
knapsack. Maximize ?i1..nfivi
Maximize ?i1..nfivi Subject to
?i1..nfiwiC, Subject to
?i1..nfiwiC, 0fi1,i1..n
fi e 0,1, i 1..n Fractional Knapsack
Problem Integer Knapsack Problem
21
Example 2. The knapsack problem
The Fractional Knapsack Problem can be solved
optimally using Greedy 3. The Integer Knapsack
Problem is NP-complete, so unlikely to be
solvable in polynomial time unless PNP.
22
Example 3. Activity-Selection
Given S1,2,,n activities, each activity i
has a start time si and a finish time fi, si fi
. Once i is selected, it must be served in
interval si,fi). We cant serve two activities
if they are incompatible (i.e., their service
intervals intersect). Question How to select a
maximum-size set of mutually compatible
activities?
23
Example 3. Activity-Selection
Given S1,2,,n activities, each activity i
has a start time si and a finish time fi, si fi
. Once i is selected, it must be served in
interval si,fi). We cant serve two activities
if they are incompatible (i.e., their service
intervals intersect). Question How to select a
maximum-size set of mutually compatible
activities?
24
Example 3. Activity-Selection
Idea Order the activities by increasing
finishing time, i.e., f1f2fn
f9
f3
f6
f5
f2
f10
f1
f7
f8
f4
25
Example 3. Activity-Selection

• Idea Order the activities by increasing
  finishing time, i.e., f1f2fn
• Greedy-Activity-Selector(s,f)
• 1. n ? lengths
• 2. A ? 1
• 3. j ? 1
• 4. for i 2 to n
• If si fj
• then A ? A U i
• j ? i
• 8. Return A

f9
f3
f6
f2
f5
f10
f1
f7
f8
f4
26
Example 3. Activity-Selection

• Idea Order the activities by increasing
  finishing time, i.e., f1f2fn
• Greedy-Activity-Selector(s,f)
• 1. n ? lengths
• 2. A ? 1
• 3. j ? 1
• 4. for i 2 to n
• If si fj
• then A ? A U i
• j ? i
• 8. Return A

f9
f3
f6
f2
f5
f10
f1
f7
f8
f4
27
Example 3. Activity-Selection

• Idea Order the activities by increasing
  finishing time, i.e., f1f2fn
• Greedy-Activity-Selector(s,f)
• 1. n ? lengths
• 2. A ? 1
• 3. j ? 1
• 4. for i 2 to n
• If si fj
• then A ? A U i
• j ? i
• 8. Return A

f9
f3
f6
f2
f5
f10
f1
f7
f8
f4
28
Example 3. Activity-Selection

• Idea Order the activities by increasing
  finishing time, i.e., f1f2fn
• Greedy-Activity-Selector(s,f)
• 1. n ? lengths
• 2. A ? 1
• 3. j ? 1
• 4. for i 2 to n
• If si fj
• then A ? A U i
• j ? i
• 8. Return A

f9
f3
f6
f2
f5
f10
f1
f7
f8
f4
29
Example 3. Activity-Selection

• Idea Order the activities by increasing
  finishing time, i.e., f1f2fn
• Greedy-Activity-Selector(s,f)
• 1. n ? lengths
• 2. A ? 1
• 3. j ? 1
• 4. for i 2 to n
• If si fj
• then A ? A U i
• j ? i
• 8. Return A

f9
f3
f6
f2
f5
f10
f1
f7
f8
f4
30
Example 4. Huffman codes
Motivation You have a 100,000-character data
file F, with only 6 characters a,b,c,d,e,f. You
want to have a way to encode them to save space
(remember that at the bottom-most level,
everything is binary). a b c d e f Frequency 4
5000 13000 12000 16000 9000 5000 Fixed-length 000
001 010 011 100 101 codeword
31
Example 4. Huffman codes
Motivation You have a 100,000-character data
file F, with only 6 characters a,b,c,d,e,f. You
want to have a way to encode them to save space
(remember that at the bottom-most level,
everything is binary). a b c d e f Frequency 4
5000 13000 12000 16000 9000 5000 Fixed-length 000
001 010 011 100 101 codeword Variable-length 0 10
1 100 111 1101 1100 codeword
32
Example 4. Huffman codes
Motivation You have a 100,000-character data
file F, with only 6 characters a,b,c,d,e,f. You
want to have a way to encode them to save space
(remember that at the bottom-most level,
everything is binary). a b c d e f Frequency 4
5000 13000 12000 16000 9000 5000 Fixed-length 000
001 010 011 100 101 codeword Cost 100,000 x 3
300,000 bits Variable-length 0 101 100 111 1101 1
100 codeword
33
Example 4. Huffman codes
Motivation You have a 100,000-character data
file F, with only 6 characters a,b,c,d,e,f. You
want to have a way to encode them to save space
(remember that at the bottom-most level,
everything is binary). a b c d e f Frequency 4
5000 13000 12000 16000 9000 5000 Fixed-length 000
001 010 011 100 101 codeword Cost 100,000 x 3
300,000 bits Variable-length 0 101 100 111 1101 1
100 codeword Cost 45000x113000x312000x316000x3
9000x45000x4 224,000 bits
34
Example 4. Huffman codes
Motivation You have a 100,000-character data
file F, with only 6 characters a,b,c,d,e,f. You
want to have a way to encode them to save space
(remember that at the bottom-most level,
everything is binary). So we want to design an
optimal variable-length codes.
35
Example 4. Huffman codes
Motivation You have a 100,000-character data
file F, with only 6 characters a,b,c,d,e,f. You
want to have a way to encode them to save space
(remember at the bottom-most level, everything is
binary). So we want to design an optimal
variable-length codes. Prefix-free codes no
codeword is a prefix of some other codeword. Easy
to encode and decode, no ambiguity. Example.
c?d?f?a100?111?1100?010011111000
36
Example 4. Huffman codes
Prefix-free codes no codeword is a prefix of
some other codeword. Easy to encode and decode,
no ambiguity. Example. c?d?f?a100?111?1100?0100
11111000 We usually use a binary tree to
represent the prefix-free codes, its leaves are
the given characters. The binary codeword for a
character is the path from the root to it, where
0 means go to left child and 1 means go to right
child.
37
Example 4. Huffman codes
100
100
0
1
0
1
55
45
a
86
14
0
1
1
0
0
30
58
28
14
25
0
1
1
0
0
1
1
0
1
14
16
45
9
5
b
13
12
16
12
13
c
d
1
b
a
0
e
d
c
f
9
5
f
e

• Fixed-length codeword

• Variable-length codeword

38
Example 4. Huffman codes

• Huffman(C)
• n ? C
• Q ? C //Q is a priority queue, keyed on
  frequency f
• for i1 to n-1
• z ? Allocate-node()
• leftz?x?Extract-Min(Q)
• rightz?y?Extract-Min(Q)
• fz ? fxfy
• Insert(Q,z)
• Return Extract-Min(Q) //now we have the binary
  tree

39
Example 4. Huffman codes

• Huffman(C)
• n ? C
• Q ? C //Q is a priority queue, keyed on
  frequency f
• for i1 to n-1
• z ? Allocate-node()
• leftz?x?Extract-Min(Q)
• rightz?y?Extract-Min(Q)
• fz ? fxfy
• Insert(Q,z)
• Return Extract-Min(Q) //now we have the binary
  tree
• What is the running time?

40
Example 4. Huffman codes

• Huffman(C)
• n ? C
• Q ? C //Q is a priority queue, keyed on
  frequency f
• for i1 to n-1
• z ? Allocate-node()
• leftz?x?Extract-Min(Q)
• rightz?y?Extract-Min(Q)
• fz ? fxfy
• Insert(Q,z)
• Return Extract-Min(Q) //now we have the binary
  tree
• What is the running time? O(n log n)!