MCS 101: Design and Analysis of Algorithms

1
MCS 101 Design and Analysis of Algorithms

• Instructor
• Neelima Gupta
• ngupta_at_cs.du.ac.in

2
Dynamic Programming

• Instructor Ms. Neelima Gupta

3
Interval/Job Scheduling a greedy approach
4
Weighted Interval Scheduling

• Each jobi has (si , fi , pi)
• starting time si ,
• finishing time fi, and
• profit pi
• Aim Find an optimal schedule of job that makes
  the maximum profit.
• Greedy approach choose the job which finishes
  first..does not work.
• So, DP

5
i Si Fi Pi

• 2 2 4 3
• 1 1 5 10
• 3 4 6 4
• 4 5 8 20
• 5 6 9 2

6

Weighted Interval Scheduling
P(1)10
P(2)3
P(3)4
P(4)20
P(5)2
Time
0
1
7
2
3
4
5
6
8
9
7
Greedy Approach

P(1)10
P(2)3
P(3)4
P(4)20
P(5)2
Time
0
1
7
2
3
4
5
6
8
9
.
8
Greedy does not work

P(1)10
P(2)3
Optimal schedule
P(3)4
P(4)20
Schedule chosen by greedy app
P(5)2
Time
0
1
7
2
3
4
5
6
8
9
Greedy approach takes job 2, 3 and 5 as best
schedule and makes profit of 7. While optimal
schedule is job 1 and job4 making profit of 30
(1020). Hence greedy will not work
9
DP Solution for WIS

• Let mj optimal schedule solution from the
  first jth jobs, (jobs are sorted in the
  increasing order of their finishing times)
• pjprofit of jth job.
• pj largest index iltj , such that
  interval i and j are disjoint i.e. i is the
  rightmost interval that ends before j begins or
• the last interval compatible
  with j and is before j.

10
DP Solution for WIS

• Either j is in the optimal solution or it is not.
• If it is, then mj pj profit when
  considering the jobs before and including the
  last job compatible with j
• i.e. mj pj mp(j)
• If it is not, then mj mj-1
• Thus,
• mj max(pj mp(j), mj-1)

11
Recursive Paradigm

• Write a recursive function to compute
  mnafterall we are only interested in that.
• some of the problems are solved several times
  leading to exponential time.

12
INDEX 1 2 3 4 5 6
P10
V12
V24
V34
V47
V52
V61
13
INDEX 1 2 3 4 5 6
P10
V12
P20
V24
V34
V47
V52
V61
14
INDEX 1 2 3 4 5 6
P10
V12
P20
V24
P31
V34
V47
V52
V61
Thanks to Nikita Khanna(19)
15
INDEX 1 2 3 4 5 6
P10
V12
P20
V24
P31
V34
P40
V47
V52
V61
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
16
INDEX 1 2 3 4 5 6
P10
V12
P20
V24
P31
V34
P40
V47
V52
P53
V61
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
17
INDEX 1 2 3 4 5 6
P10
V12
P20
V24
P31
V34
P40
V47
V52
P53
V61
P63
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
18
Recursive Paradigm tree of recursion for WIS
19
Recursive Paradigm

• Sometimes some of the problems are solved several
  times leading to exponential time. For example
  Fibonacci numbers.
• DP Solution Solve a problem only once and use it
  as and when required
• Top-down DP ..Memoization.generally storage
  cost is large
• Bottom-Up DP .Iterative

20
Principles of DP

• Recursive Solution (Optimal Substructure
  Property)
• Overlapping Subproblems
• Total Number of Subproblems is polynomial

21
mj maxmj-1, mpj pj
m 0 1 2
3 4 5
6
Thanks to Nikita Khanna(19)
22
mj maxmj-1, mpj pj
0
m 0 1 2
3 4 5
6
Thanks to Nikita Khanna(19)
23
mj maxmj-1, mpj pj
Max0,02
0
2
m 0 1 2
3 4 5
6
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
24
mj maxmj-1, mpj pj
Max0,02
Max2,04
0
2
4
m 0 1 2
3 4 5
6
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
25
mj maxmj-1, mpj pj
Max0,02
Max2,04
0
2
4
6
m 0 1 2
3 4 5
6
Max4,24
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
)
26
mj maxmj-1, mpj pj
Max0,02
Max6,07
Max2,04
0
2
4
6
7
m 0 1 2
3 4 5
6
Max4,24
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
27
mj maxmj-1, mpj pj
Max0,02
Max6,07
Max2,04
0
2
4
6
7
8
m 0 1 2
3 4 5
6
Max7,62
Max4,24
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
28
mj maxmj-1, mpj pj
Max0,02
Max8,61
Max6,07
Max2,04
0
2
4
6
7
8
8
m 0 1 2
3 4 5
6
Max7,62
Max4,24
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
Thanks to Nikita Khanna(19)
29
Matrix Chain Multiplication

• Definition-
• are the matrix of order
• respectively.
• e.g.
  are the matrix of order
• (2 x 3), (3 x 4) , ( 4 x 5)
  respectively.
• 
  (2 x 3 x 4 ) (2 x 4 x 5)
• TOTAL
  MULTIPLICATION 24 40 64
• 
  (3 x 4 x 5) (2 x 3x 5) 90
• So, we have seen that by changing the evaluation
  sequence cost of operation also changes.

THANKS TOOM JI(26)
30
Recursive Solution

• In general,

31
Overlapping Subproblems

• m14
• m1,m2,3,4 m1,2,m3,4
  m1,2,3,m4
• m2,m3,4 m2,3,m4 m1,m2,3
  m1,2,m3

THANKS TO OM JI(26)
32
Number of sub-problems

• Mij
  i lt j
• n choices n choices
• Number of subproblems n2 ..polynomial

THANKS TO OM JI(26)
33
Example

• Matrix dimensions
• A1 3 5
• A2 5 4
• A3 4 2
• A4 2 7
• A5 7 3
• A6 3 8

THANKS TO OM JI(26)
34
Problem of parenthesization
A1 A2 A3 A4 A5 A6
A1 0 60
A2 - 0 40
A3 - - 0 56
A4 - - - 0 42
A5 - - - - 0 168
A6 - - - - - 0
A1 A2 354 60 A2A3 542 40 A3A4
427 56 A4A5 273 42 A5A6 738
168
THANKS TO OM JI(26)
35
Problem of parenthesization

• A1 _A3 A1 A2 A3 min( (A1.A2).A3 60 342
  84 or A1.(A2 .A3)40 35270)
• 70
• A2 _A4 A2 A3 A4 min( (A2.A3).A4 40
  527110 or A2.(A3.A4)56 547
  196 )
• 110
• A3 _A5 A3 A4 A5 min( (A3 .A4).A5
  56473140 or A3.(A4 .A5)42 423 66)
  66
• A4 _A6 A4 A5 A6 min(
• (A4 A5 )A6 4223890 or
• A4 (A5 A6 )168 278 312 ) 90

A1 A2 A3 A4 A5 A6
A1 0 60 70
A2 - 0 40 110
A3 - - 0 56 66
A4 - - - 0 42 90
A5 - - - - 0 168
A6 - - - - - 0
THANKS TO OM JI(26)
36
Problem of parenthesization
A1 A2 A3 A4 A5 A6
A1 0 60 70 K1 112 K3 130 K3 202 K5
A2 - 0 40 110 K3 112 K3 218 K3
A3 - - 0 56 66 K3 154 K3
A4 - - - 0 42 90 K5
A5 - - - - 0 168
A6 - - - - - 0
THANKS TO OM JI(26)
37
Running Time
?(n2) entries each takes O(n) time to compute The
running time of this procedure is ?(n3).
Thanks to OM JI(26)
38
Segmented Least Squares Multi-way Choices
39

• Given a set P of n points in a plane denoted by
  (x1,y1), (x2,y2), . . . , (xn,yn) and line L
  defined by the equation y ax b, error of line
  L with respect to P is the sum of squares of the
  distances of these points from L.
• Error(L, P) Parila..
• A
  line of best fit

40
Line of Best Fit

• Line of Best Fit is the one that minimizes this
  error. This can be computed in O(n) time using
  the following formula
• Parila
• The formula is obtained by differentiating the
  error wrt a and equating to zero and wrt b and
  equating to zero

41

• Consider the following scenario Clearly
  approximating these points with a single line is
  not a good idea.
• A set of points that lie approximately
  on two lines
• Using two lines clearly gives a better
  approximation.

42
(contd.)

• A set of points that lie
  approximately on three lines
• In this case, 3 lines will give us a better
  approximation.

43
Segmented Least Square Error

• In all these cases, we are able to tell the
  number of lines we must use by looking at the
  points. In general, we dont know this number.
  That is we dont know what is the minimum number
  of lines we must use to get a good approximation.
  
• Thus our aim becomes to minimize the number of
  lines that minimize the least square error.

44
DESIGNING THE ALGORITHM

• We know that the last point p n belongs to a
  single segment in the optimal partition and this
  segment begins at some point, say pi .
• Thus, if we know the identity of the last
  segment, then we can recursively solve the
  problem on the remaining points pi pi-1 as
  illustrated below

45
WRITING THE RECURRENCE

• If the last segment in the optimal is pi , ,
  pn, then the value of optimal is,
• SLS(n) SLS(i-1) c ein
• where c is the cost of using the segment and ein
  is the least square error of this segment.
• But since we dont know i, well compute it as
  follows
• SLS(n) mini SLS(i-1)
  c eij
• General recurrence
• For the sub-problem p1, , pj ,
• SLS(j) mini SLS(i-1) c
  eij

46
Time Analysis

• eij values can be pre-computed in O(n3) time.
• Additional Time n entries, each entry computes
  the minimum of at most n values each of which can
  be computed in constant time. Thus a total of
  O(n2).

47
FRACTIONAL KNAPSACK PROBLEM

• Given a set S of n items, with value vi and
  weight wi and a knapsack with capacity W.
• Aim Pick items with maximum total value but
  with weight at most W. You may choose fractions
  of items.

48
GREEDY APPROACH

• Pick the items in the decreasing order of value
  per unit weight i.e. highest first.

49
Example

• knapsack capacity 50
• Item 2 item 3
• Item 1
• vi 60 vi 100 vi 120
• vi/ wi 6 vi/ wi 5 vi/
  wi 4

30
20
10
Thanks to Neha Katyal
50
Example

• knapsack capacity 50
• Item 2 item 3
• 
  
  
• 
  
  60
• vi 100 vi 120
• vi/ wi 5 vi/ wi 4

30
20
10
Thanks to Neha Katyal
51
Example

• knapsack capacity 50
• item 3
• 
  
  100
• 
  
  
  
• 
  
  60
• vi 120
• vi/ wi 4

20
30
10
Thanks to Neha Katyal
52
Example

• knapsack capacity 50
• 
  80
• 
  
• 
  
  100
• 
  
  
  
• 
  
  60
• 
  240

20/30
20
10
Thanks to Neha Katyal
53
0-1 Kanpsack

• example to show that the above greedy approach
  will not work,
• So, DP

54
GREEDY APPROACH DOESNT WORK FOR 0-1 KNAPSACK

• Counter Example
• knapsack
• Item 2 item 3
• Item 1
• vi 60 vi 100 vi 120
• vi/ wi 6 vi/ wi 5 vi/
  wi 4

30
20
10
Thanks to Neha Katyal
55
Example

• knapsack
• Item 2 item 3
• 
  
  
• 
  
  60
• vi 100 vi 120
• vi/ wi 5 vi/ wi 4

30
20
10
Thanks to Neha Katyal
56
Example

• knapsack
• item 3
• 
  
  100
• 
  
  
  
  
• 
  
  60
• vi 120
  160
• vi/ wi 4
• 
  suboptimal

20
30
10
Thanks to Neha Katyal
57
DP Solution for 0-1KS

• Let mI,w be the optimal value obtained when
  considering objects upto I and filling a knapsack
  of capacity w
• m0,w 0
• mi,0 0
• mi,w mi-1,w if wi gt w
• mi,w maxmi-1, w-wi vi , mi-1, w if wi
  lt w
• Running Time Pseudo-polynomial

58
Example

• n 4
• W 5
• Elements (weight, value)
• (2,3), (3,4), (4,5), (5,6)

59
(2,3), (3,4), (4,5), (5,6)
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0
0
0
0
60
(2,3), (3,4), (4,5), (5,6)
As wltw1 m1,1 m1-1,1 m0,1
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0
0
0
0
61
(2,3), (3,4), (4,5), (5,6)
As wltw2 m2,1 m2-1,1 m1,1
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0
0 0
0
0
62
(2,3), (3,4), (4,5), (5,6)
As wltw3 m3,1 m3-1,1 m2,1
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0
0 0
0 0
0
63
(2,3), (3,4), (4,5), (5,6)
As wltw4 m4,1 m4-1,1 m3,1
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0
0 0
0 0
0 0
64
(2,3), (3,4), (4,5), (5,6)
As wgtw1 m1,2 maxm1-1,2 ,
m1-1,2-23 max 0,03
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3
0 0
0 0
0 0
65
(2,3), (3,4), (4,5), (5,6)
As wltw2 m2,2 m2-1,2 m1,2
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3
0 0 3
0 0
0 0
66
(2,3), (3,4), (4,5), (5,6)
As wltw3 m3,2 m3-1,2 m2,2
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3
0 0 3
0 0 3
0 0
67
(2,3), (3,4), (4,5), (5,6)
As wltw4 m4,2 m4-1,2 m3,2
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3
0 0 3
0 0 3
0 0 3
68
(2,3), (3,4), (4,5), (5,6)
As wgtw1 m1,3 maxm1-1,3 ,
m1-1,3-23 max 0,03
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3
0 0 3
0 0 3
0 0 3
69
(2,3), (3,4), (4,5), (5,6)
As wgtw2 m2,3 maxm2-1,3 ,
m2-1,3-34 max 3,04
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3
0 0 3 4
0 0 3
0 0 3
70
(2,3), (3,4), (4,5), (5,6)
As wltw3 m3,3 m3-1,3 m2,3
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3
0 0 3 4
0 0 3 4
0 0 3
71
(2,3), (3,4), (4,5), (5,6)
As wltw4 m4,3 m4-1,3 m3,3
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3
0 0 3 4
0 0 3 4
0 0 3 4
72
(2,3), (3,4), (4,5), (5,6)
As wgtw1 m1,4 maxm1-1,4 ,
m1-1,4-23 max 0,03
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3
0 0 3 4
0 0 3 4
0 0 3 4
73
(2,3), (3,4), (4,5), (5,6)
As wgtw2 m2,4 maxm2-1,4 ,
m2-1,4-34 max 3,04
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3
0 0 3 4 4
0 0 3 4
0 0 3 4
74
(2,3), (3,4), (4,5), (5,6)
As wgtw3 m3,4 maxm3-1,4 ,
m3-1,4-45 max 4,05
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3
0 0 3 4 4
0 0 3 4 5
0 0 3 4
75
(2,3), (3,4), (4,5), (5,6)
As wltw4 m4,4 m4-1,4 m3,4
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3
0 0 3 4 4
0 0 3 4 5
0 0 3 4 5
76
(2,3), (3,4), (4,5), (5,6)
As wgtw1 m1,5 maxm1-1,5 ,
m1-1,5-23 max 0,03
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3 3
0 0 3 4 4
0 0 3 4 5
0 0 3 4 5
77
(2,3), (3,4), (4,5), (5,6)
As wgtw2 m2,5 maxm2-1,5 ,
m2-1,5-34 max 3,34
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3 3
0 0 3 4 4 7
0 0 3 4 5
0 0 3 4 5
78
(2,3), (3,4), (4,5), (5,6)
As wgtw3 m3,5 maxm3-1,5 ,
m3-1,5-45 max 7,05
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3 3
0 0 3 4 4 7
0 0 3 4 5 7
0 0 3 4 5
79
(2,3), (3,4), (4,5), (5,6)
As wgtw4 m4,5 maxm4-1,5 ,
m4-1,5-56 max 7,06
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3 3
0 0 3 4 4 7
0 0 3 4 5 7
0 0 3 4 5 7
80
(2,3), (3,4), (4,5), (5,6)
W 0 1 2 3 4 5
i 0 1 2 3 4
0 0 0 0 0 0
0 0 3 3 3 3
0 0 3 4 4 7
0 0 3 4 5 7
0 0 3 4 5 7
81
Pseudo-polynomial algorithm

• An algorithm that is polynomial in the numeric
  value of the input (which is actually exponential
  in the size of the input the number of digits).
• Thus O(n) time algorithm to test whether n is
  prime or not is pseudo-polynomial.
• DP solution to 0-1 Knapsack is pseudo-polynomial
  as it is polynomial in K, the capacity (one of
  the inputs) of the Knapsack.

82
Weakly/Strongly NPC problems

• An NP complete problem with a known
  pseudo-polynomial solution is stb weakly NPC.
• An NPC problem for which it has been proved that
  it cannot admit a pseudo-polynomial solution
  unless P NP is stb strongly NPC.

83
Sequence Alignment

• SUBMITTED BY
• BY PULKIT OHRI(29)

84
STRING SIMILARITY

• ocurrance
• occurrence
• 
  THANKS
  TO PULKIT

o
c
u
r
r
a
n
c
e
-
c
c
u
r
r
e
n
c
e
o
6 mismatches, 1 gap
o
c
u
r
r
a
n
c
e
-
c
c
u
r
r
e
n
c
e
o
1 mismatch, 1 gap
o
c
u
r
r
n
c
e
-
-
a
c
c
u
r
r
n
c
e
o
e
-
0 mismatch, 3 gaps
85

• PROBLEM Given two strings X x1 x2 . . . xn
  and Y y1 y2 . . .. ym
• GOAL Find an alignment of minimum cost, where
  d is the cost of a gap and ? is the cost of a
  mismatch.
• Let OPT(i,j)min cost of aligning strings X x1
  x2 . . . xi and Y y1 y2 . . .yj
• 
  
  THANKS TO PULKIT

86

• 4 cases
• Case1 xi is aligned with yj in OPT
  and they match
• Pay mismatch for xi yj min cost of aligning
  two strings
• Case 2 OPT does not matches xi.
• Pay gap for xi min cost of aligning x1 x2 . . .
  xi-1 and y1 y2 . . . yj
• Case 3 OPT does not matches yj.
• Pay gap for yj min cost of aligning x1 x2 . . .
  xi and y1 y2 . . . Yj
• 
  THANKS TO PULKIT

87
(No Transcript)
88

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• 
  
  
  THANKS TO PULKIT

0
89

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• Advance name but not mean.So,pay gap cost d2.
• 
  THANKS TO PULKIT

0 2
90

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• d224
• 
  
  
  
  THANKS TO PULKIT

0 2 4
91

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• d 426
• 
  
  
  
  THANKS TO PULKIT

0 2 4 6
92

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• d 628
• 
  
  
  
  THANKS TO PULKIT

0 2 4 6 8
93

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• 
  
  
  
  THANKS TO PULKIT

8
6
4
2
0 2 4 6 8
94

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• Advance both name and mean and pay mismatch cost
  ?1
• -m
• -n

8
6
4
2 (1)
0 2 4 6 8
95

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• Advance mean,pay gap cost d224.
• - n -
• - - m
• 0,2,2

8
6
4
2 (4)
0 2 4 6 8
96

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• Advance name,pay gap cost cost d224
• - m -
• - - n
• 0,2,2

8
6
4
2 (4)
0 2 4 6 8
97

• EG let d2 and ?03
• mean
• name
• n
• a
• e
• m
• n a
  m e
• So , took the min. 1
• 
  
  
  
  THANKS TO PULKIT

8
6
4
2 1
0 2 4 6 8
98

• Similarly,
• n
• a
• e
• m
• -
• - n a m
  e
• RUNNING TIME ?(mn)
• 
  
  
  
  THANKS TO PULKIT

8 6 5 4 6
6 5 3 5 5
4 3 2 4 4
2 1 3 4 6
0 2 4 6 8
99
Thank You!