ALGORITHM TYPES - PowerPoint PPT Presentation

Loading...

PPT – ALGORITHM TYPES PowerPoint presentation | free to download - id: 68522b-M2I0Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

ALGORITHM TYPES

Description:

ALGORITHM TYPES Greedy, Divide and Conquer, Dynamic Programming, and Backtracking Note the general strategy from the examples The classification is neither exhaustive ... – PowerPoint PPT presentation

Number of Views:41
Avg rating:3.0/5.0
Date added: 13 March 2020
Slides: 45
Provided by: Debasi3
Learn more at: http://cs.fit.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: ALGORITHM TYPES


1
ALGORITHM TYPES
  • Greedy, Divide and Conquer, Dynamic Programming,
    and Backtracking
  • Note the general strategy from the examples
  • The classification is neither exhaustive (there
    may be other types), nor mutually exclusive
    (one may combine)

2
PROBLEM 1 DYNAMIC PROGRAMMING STRATEGY
  • In case the divide and conquer strategy can
    divide the problem at a very small level, and
    there are repetition of some calculations over
    some components
  • One can apply a bottom up approach calculate the
    smaller components first and then keep combining
    them until the highest level of the problem is
    solved.
  • Draw the recursion tree of Fibonacci-series
    calculation, you will see example of such
    repetitive calculations
  • f(n) f(n-1) f(n-2), for ngt1 and f(n)1
    otherwise
  • fib(n) calculation
  • n 1, 2, 3, 4, 5
  • fib 1, 2, 3, 5, 8

3
DYNAMIC PROGRAMMING STRATEGY (Continued)
  • Recursive fib(n)
  • if (nlt1) return 1
  • else return (fib(n-1) fib(n-2)).
  • Time complexity exponential, O(kn) for some
    kgt1.0
  • Iterative fibonacci(n)
  • fib(0) fib(1) 1
  • for i2 through n do
  • fib(i) fib(i-1) fib(i-2)
  • end for
  • return fib(n).
  • Time complexity O(n), Space complexity O(n)

4
DYNAMIC PROGRAMMING STRATEGY (Continued)
  • SpaceSaving-fibonacci(n)
  • if (nlt1) return 1
  • int last1, last2last1, result1
  • for i2 through n do
  • result last last2last
  • last2last last
  • lastresult
  • end for
  • return result.
  • Time complexity O(n), Space complexity O(1)

5
DYNAMIC PROGRAMMING STRATEGY (Continued)
  • The recursive call recalculates fib(1) 5 times,
    fib(2) 3 times, fib(3) 2 times - in fib(5)
    calculation. The complexity is exponential.
  • In iterative calculation we avoid repetition by
    storing the needed values in variables -
    complexity of order n.
  • Dynamic Programming approach consumes more memory
    to store the result of calculations of lower
    levels for the purpose of calculating the next
    higher level. They are typically stored in a
    table.

6
Problem 2 0-1 Knapsack Problem (not in the book)
  • Given a set of objects with (Weight, Profit)
    pair, and a Knapsack of limited weight capacity
    (M), find a subset of objects for the knapsack to
    maximize total profit P
  • Sample Problem
  • Objects (wt, p) (2, 1), (3, 2), (9, 20), (5,
    3). M8
  • Exhaustive Algorithm Try all subset of objects.
  • How many?

total(0 lbs, 0) (2,1) total (2lbs,
1) (9,20) Illegal, total wtgt8
(2,1), (3,2) (5lbs, 3)
Total possibilities 2n For each object
present or absent 0 or 1 A bit string is a
good representation for a set, e.g.
000101100. n-bits
7
0-1 Knapsack Problem
  • DP uses a representation for optimal profit P(j,
    k)
  • for the first j objects considered
  • (in any arbitrary ordering of objects)
  • with a variable knapsack limit k
  • Develop the table for P(j, k)
  • with rows j 1 n (objects), and
  • for each row, go from k 0 M (the KS limit)
  • Finally P(n, M) gets the result

w-gt 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0
O1 0 0 1 1 1 gt 1
O1 O2 0 0 1 2 2 3 3 gt 3
O1 O2 O3 0 0 1 2 2 3 3 4 5
8
0-1 Knapsack Recurrence for computing P
  • The recurrence, for all k0M, j0N
  • P(j, k) (case 1) P(j-1, k),
  • if wjgtk, wj is weight of the j-th object
  • else
  • P(j, k) (case2) max P(j-1, k-wj)pj,
  • P(j-1, k)
  • Explanation for the formula is quite intuitive
  • Recurrence terminates
  • P(0, k)0, P(j, 0)0, for all ks and js

Objects (wt, p) (2, 1), (3, 2), (5, 3).
M9
w-gt 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0
O1 0 0 1 1 1 1 1 1 1
O1 O2 0 0 1 2 2 3 3 3 3
O1 O2 O3 0 0 1 2 2 P(3,5)3
9
Recurrence -gt recursive algorithm
  • Algorithm P(j, k)
  • Input Set of n objects with (wi, pi) pairs, and
    knapsack limit M
  • Output Maximum profit P for subset of objects
    with total wt ltM
  • If j lt 0 or k lt 0 then return 0 //recursion
    termination
  • else
  • if wjgtk then
  • return P(j-1, k)
  • else
  • return maxP(j-1, k-Wj) pj, P(j-1, k)
  • End algorithm.
  • Driver call P(n, M), for given n objects KS
    limit M.
  • This is not table building Ps are recursive
    functions

Complexity ?
Exponential O(2n)
Why?
Same reason as in Fibonacci number
calculation Repeats computing Ps
10
0-1 Knapsack Recursive call tree
  • Objects (wt, p) (2, 1), (3, 2), (5, 3).
    M8

P(3,8)
P(2,8-5)
P(2,8)
P(1,8-3)
P(1,3-3)
P(1,8)
P(1,3)
w1 2gt0, not called
P(0,8)
P(0,3-2)
P(0,3)
P(0,5-2)
P(0,5)
P(0,8-2)
P(0,0-2)
P(0,0)
Each leaf has w0, and P(0,) returns a value, 0
to the caller above
w-gt 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0
O1 0 0 1 1 1 gt 1
O1 O2 0 0 1 2 2 3 3 gt 3
O1 O2 O3 0 0 1 2 2 3 3 4 5
11
0-1 Knapsack DP-algorithm
Do not repeat the same computation, store them in
a table and reuse
  • Algorithm P(j, k)
  • Input Set of n objects with (wi, pi) pairs, and
    knapsack limit M
  • Output Maximum profit P for a subset of objects
    with total wt ltM
  • For all k, P(0, k) 0 For all j, P(j, 0) 0
    //initialize
  • For j1 to n do
  • For k 1 to M do
  • if wjgtk then
  • P(j, k) P(j-1, k)
  • else
  • P(j, k) maxP(j-1, k-Wj) pj, P(j-1, k)
  • End loops and algorithm

Complexity O(nM), pseudo-polynomial because M is
an input value. If M30.5 the table would be of
size O(10nM), or if M30.54 it would be O(100NM).
12
0-1 Knapsack Problem (Example)
  • Objects (wt, p) (2, 1), (3, 2), (5, 3).
    M8

w-gt 0 1 2 3 4 5 6 7 8
0 0 0 0 0 0 0 0 0
O1 0 0 1 1 1 gt 1
O1 O2 0 0 1 2 2 3 3 gt 3
O1 O2 O3 0 0 1 2 2 3 3 4 5
2
0
-5 lbs
(23)
13
What if M9
  • Objects (wt, p) (2, 1), (3, 2), (5, 3). M9

w-gt 0 1 2 3 4 5 6 7 8 9
0 0 0 0 0 0 0 0 0 0
O1 0 0 1 1 1 1 1 1 1 1
O1 O2 0 0 1 2 2 3 3 3 3 3
O1 O2 O3 0 0 1 2 2 3 3 4 5 5
-5 lbs
(23)
HOW TO FIND KNAPSACK CONTENT FROM TABLE? SPACE
COMPLEXITY?
14
Memoisation algorithm 0-1 knapsack
  • Algorithm P(j, k)
  • If j lt 0 or k lt 0 then return 0 //recursion
    termination
  • else
  • if Wjgtk then
  • yM(j-1, k) if ylt0 y P(j-1, k) M(j-1,
    k)y // P( ) is a recursive call
  • return y
  • else
  • xM(j-1, k-Wj) if xlt0 x P(j-1, k-Wj)
    M(j-1, k-Wj)x
  • yM(j-1, k) if ylt0 y P(j-1, k) M(j-1,
    k)y
  • M(j, k) maxx pj, y
  • return maxx pj, y
  • End algorithm.
  • Driver Initialize a global matrix M(0-gtn, 0-gtM)
    with -1 call P(n, M)
  • Complexity ?

15
Problem 3 Ordering of Matrix-chain
Multiplications
  • A chain of matrices to be multiplied ABCD,
  • dimensions A (5x1), B(1x4), C(4x3), and D(3x6).
  • Resulting matrix will be of size (5x6).
  • scalar (or integer) multiplications for (BC) is
    1.4.3,
  • and the resulting matrixs dimension is (1x3), 1
    column and 3 rows.

16
Ordering of Matrix-chain Multiplications
  • A (5x1), B(1x4), C(4x3), and D(3x6)
  • There are multiple ways to order the
    multiplication (A(BC))D, A(B(CD)), (AB)(CD),
    ((AB)C)D, and A((BC)D)
  • note Resulting matrix would be the same
  • but the efficiency of calculation may vary
    drastically
  • Efficiency depends on scalar multiplications
  • In the case of (A(BC))D, it is 1.4.3 5.1.3
    5.3.6 117
  • In the case (A(B(CD)), it is 4.3.6 1.4.6
    5.1.6 126
  • Our problem here is to find the best such
    ordering
  • An exhaustive search over all orderings is too
    expensive - Catalan number involving n!

17
Recurrence for Ordering of Matrix-chain
Multiplications
  • For a sequence A1...(Aleft....Aright)...An, we
    want to find the optimal break point for the
    parenthesized sequence.
  • Calculate for ( r-l1) number of cases and find
    the minimum min(Al...Ai) (Ai1... Ar), with l?
    i lt r,
  • Recurrence for optimum scalar-multiplication
  • M(l, r)
  • minM(l, i) M(i1, r) rowl .coli .colr,
  • with l? i lt r.

r 1 2 3 4 lft 1 0 15 35(2) 69(2) 2 0 12
42 3 0 24 4 0
Sample M(i,j)
18
Matrix-chain Recursive algorithm
  • Recursive Algorithm M(l, r)
  • if rlt l then return 0 // Recurrence
    termination no scalar-mult
  • else
  • return minM(l, i) M(i1, r) rowl.coli
    .colr ,
  • for l?iltr
  • end algorithm.
  • Driver call M(1, n) for the final answer.

19
Recurrence for Ordering of Matrix-chain
Multiplications to Bottom-up Computation
  • Recurrence for optimum scalar-multiplication
  • M(l, r)
  • minM(l, i) M(i1, r) rowl .coli .colr,
  • with l? i lt r.
  • To compute M(l,r), you need M(l,i) and M(i1,r)
    available for ALL is
  • E.g., For M(3,9), you need M(3,3), M(4,9), and
    M(3,4), M(5,9),
  • Need to compute smaller size Ms first
  • Gradually increase size from 1, 2, 3, , n

20
Strategy for Ordering of Bottom-up Computation
  • Recurrence for optimum number of
    scalar-multiplications
  • M(l, r)
  • minM(l, i) M(i1, r) rowl .coli .colr
    ,
  • with l? i lt r.
  • Compute by increasing size r-l11, r-l12,
    r-l13, r-l1n
  • Start the calculation at the lowest level with
    two matrices, AB, BC, CD etc.
  • Really?
  • Where does the recurrence terminate?
  • Then calculate for the triplets, ABC, BCD etc.
  • And so on

0, for lr
21
Ordering of Matrix-chain Multiplications (Example)
  • r 1 2 3 4
  • l
  • 1 0 15 35(2) 69(2)
  • 2 0 12 42
  • 3 0 24
  • 4 0

Triplets
Pairs
Singlet's
22
Matrix-chain DP-algorithm
  • Input list of pairwise dimensions of matrices
  • Output optimum number of scalar multiplications
  • for all 1? i ? n do M(i,i) 0 // diagonal
    elements 0
  • for size 2 to n do // size of subsequence
  • for l 1 to n-size1 do
  • r lsize-1 //move along diagonal
  • M(l,r) infinity //minimizer
  • for i l to r-1 do
  • x M(l, i) M(i1, r)
  • rowl .coli .colr
  • if x lt M(l, r) then M(l, r) x
  • End.
  • // Complexities?

23
Ordering of Matrix-chain Multiplications (Example)
A1 (5x3), A2 (3x1), A3 (1x4), A4 (4x6).
  • r 1 2 3 4
  • l
  • 1 0
  • 2 0
  • 3 0
  • 4 0

15
35(2)
69(2)
12
42(?)
Triplets
24
Pairs
How do you find out the actual matrix ordering?
Calculation goes diagonally. COMPLEXITY?
24
DP Ordering of Matrix-chain Multiplications
(Example)
  • A1 (5x3), A2 (3x1), A3 (1x4), A4 (4x6).
  • M(1,1) M(2, 2) M(3, 3) M(4, 4) 0
  • M(1, 2) M(1,1) M(2,2) 5.3.1 0 0 15.
  • M(1,3) mini1 M(1,1)M(2,3)5.3.4, i2
    M(1,2)M(3,3)5.1.4
  • min72, 35 35(2)
  • M(1,4) min i1 M(1,1)M(2,4)5.3.6, i2
    M(1,2)M(3,4)5.1.6,
  • i3 M(1,3)M(4,4)5.4.6
    min132, 69, 155 69(2)
  • 69 comes from the break-point i2
    (A1.A2)(A3.A4)
  • You may need to recursively break the sub-parts
    too, by looking at which value of i gave the min
    value at that stage, e.g., for (1,3) it was i2
    (A1.A2)A3

25
Ordering of Matrix-chain Multiplications (Example)
A1 (5x3), A2 (3x1), A3 (1x4), A4 (4x6).
  • r 1 2 3 4
  • l
  • 1 0 15 35(2) 69(2)
  • 2 0 12 42
  • 3 0 24
  • 4 0

Triplets
Pairs
For a chain of n matrices, Table sizeO(n2),
computing for each entryO(n) COMPLEXITY O(n2
n) O(n3)
A separate matrix B(i,j) keeping track of optimum
i, for actual matrix ordering
26
Computing the actual break points B(i,j)
r 1 2 3 4 5 6 l 1 - - (2) (2) (3) (3) 2
- - (3) (3) (4) 3 - - (4) (4) 4 - (3) (4
) 5 - - - 6 - - -
Backtrack on this table (A1..A3)(A4A6) ABCDEF
-gt (ABC)(DEF)
Then (A1..A3) -gt A1(A2 A3), (A4A6)
-gt A4(A5 A6) ABCDEF -gt (ABC)(DEF) -gt ((AB)C)
(D(EF))
27
Inductive Proof of Matrix-chain Recurrence
  • Induction base
  • if rlt l then absurd case, there is no matrix to
    multiply return 0
  • If r l, then only one matrix, no
    multiplication return 0
  • Inductive hypothesis
  • For all sizeltk, and k1, assume M(l,r) returns
    correct optimum
  • Note size r-l1
  • Inductive step, for sizek
  • Consider all possible ways to break up (lr)
    chain, for l?iltr
  • Make sure to compute the resulting pair of
    matrices multiplication rowl.coli .colr
  • Since M(l,i) and M(i1,r) are correct smaller
    size optimums, as per hypothesis, minM(l, i)
    M(i1, r) rowl.coli .colr , for l?iltr is the
    correct return value for M(l,r)

28
Be careful with the correctness of the Recurrence
behind Dynamic Programming
  • Inductive step
  • Consider all possible ways to break up (lr)
    chain, for l?iltr
  • Make sure to compute the resulting pair of
    matrices multiplication rowl.coli .colr
  • Since M(l,i) and M(i1,r) are correct smaller
    size optimums, per hypothesis, minM(l, i)
    M(i1, r) rowl.coli .colr , for l?iltr is the
    correct return value for M(l,r)
  • It is NOT always possible to combine smaller
    steps to the larger one.
  • Addition, Multiplication are associative
  • 43129 (((43) (12)) 9) , but
  • average(4,3,1,2,9) av( av(av(4,3), av(1,2)),
    9), may NOT be true
  • DP needs the correct formulation of a Recurrence
    first,
  • Then, bottom up combination such that smaller
    problems contributes to the larger ones

29
Problem 4 Optimal Binary Search Tree
  • Binary Search Problem
  • Input sorted objects, and key
  • Output the keys index in the list, or not
    found
  • Binary Search on a Tree Sorted list is organized
    as a binary tree
  • Recursively each root t is l t r, where l is
    left child and r is right child

Example 1, 3, 4, 7, 9, 10, 13, 15
1
7
3
4
10
3
7
9
13
4
2
9
10
13
15
15
30
Problem 4 Optimal Binary Search Tree
  • Problem Optimal Binary-search Tree Organization
  • Input Sorted list, with elements access
    frequency (how many times to be accessed/searched
    as a key over a period of time)
  • Output Optimal binary tree organization so that
    total cost is minimal
  • Cost for accessing each element (frequency
    access steps of the element)
  • Access steps for each node is its distance from
    the root, plus one.

Example list sorted by object order (index of A)
A1(7), A2(10), A3(5), A4(8), and A5(4)
A1(7)
A3(5)
A2(10)
A3(5)
A4(8)
A1(7)
A4(8)
A5(4)
A2(10)
A5(4)
Cost 51
72 103
82 43 77
Cost 71 102 53 84 45
94
31
Problem 4 Optimal Binary Search Tree
  • Problem Optimal Binary-search Tree Organization
  • Input Sorted list, with elements access
    frequency (how many times to be accessed/searched
    as a key over a period of time)
  • Output Optimal binary tree organization so that
    the total cost is minimal
  • Every optimization problem optimizes an objective
    function total access cost
  • Different tree organizations will have different
    aggregate access steps over all nodes,
  • because the depths of the nodes are different on
    different tree.
  • Problem We want to find the optimal aggregate
    access steps, and a bin-search tree organization
    that produces it.
  • What do you think the strategy should be?
  • Which node should be at root, and which nodes at
    leaves level?

32
Optimal Binary Search Tree (Continued)
  • Say, the optimal cost for a sub-tree is C(left,
    right).
  • Call this part as tree t
  • Note, when this sub-tree t is a branch of a root
    at higher level
  • under a parent,
  • i.e. within another higher level sub-tree h
  • then EACH nodes access step in t will increase
    by 1 in the higher level subtree h

A1 A2 Aleft Ai Aright An
Ap
Ai
t
Al
Ar
33
A1 A2 Aleft Ai Aright An
fi
C(i1, right)
C(left, i-1)
Now C(l, i-1) and C(i1, r) is no longer useful
to compute C(l, r), unless
34
Recurrence for Optimal Binary Search Tree
  • If the i-th node is chosen as the root for this
    sub-tree, then
  • C(l, r) minl? i ? r
  • f(i) C(l,
    i-1) C(i1, r)
  • åjli-1 f(j) additional cost for left
    sub-tree
  • åji1r f(j) additional cost for right
    sub-tree
  • minl ? i ? r åjlr f(j) C(l, i-1)
    C(i1, r)
  • We will use the above formula to compute C(1, n).
  • We will start from one element sub-tree (size1),
  • gradually increase size,
  • and finish with n element full tree

35
Optimal Binary Search Tree (Continued)
  • Like matrix-chain multiplication-ordering we will
    develop a triangular part of the cost matrix (rgt
    l), and
  • we will develop it diagonally (r l size)
  • Note that our boundary condition is different
    now,
  • c(l, r)0 if l gt r (meaningless cost)
  • We start from, l r single node trees
  • (not with lr-1 pairs of matrices, as in
    matrix-chain-mult case).
  • Also, i now goes from left through right, and
  • i is excluded from both the subtrees Cs.

36
Write the DP algorithm for Optimal Binary-search
Tree Organization problem
r -gt 1 2 3 4 5
l1
l2 0
l3 0 0
l4 0 0 0
l5 0 0 0 0
37
Optimal Binary Search Tree (Example)
  • Keys A1(7), A2(10), A3(5), A4(8), and A5(4).
  • Diagonals C(1, 1)7, C(2, 2)10, .
  • C(1,2)mink1 C(1,0)C(2,2)f1f201017,
  • k2 C(1,1) C(3,2) f1f2 7 0 17
    min27, 24 24 (i2)
  • j 1 2 3 4 5
  • i
  • 1 7 24(2) 34 55 67
  • 2 0 10 20 41 51
  • 3 0 5 18 26
  • 4 0 8 16
  • 5 0 4
  • This algorithm O(n3). However, O(n2) is
    feasible.

38
Problem5 All Pairs Shortest Path
  • A variation of Djikstras algorithm. Called
    Floyd-Warshals algorithm. Good for dense graph.
  • Algorithm Floyd
  • Copy the distance matrix in d1..n1..n
  • for k1 through n do //consider each vertex as
    updating candidate
  • for i1 through n do
  • for j1 through n do
  • if (dik dkj lt dij) then
  • dij dik dkj
  • pathij k // last updated via k
  • End algorithm.
  • O(n3), for 3 loops.

39
Problem 6 Gene Alignment
Score 9 matches x 1 1 mismatch x -1 1 gap x
-2 6
Find a best-score alignment
40
Problem 6 Gene Alignment
s1 s2 s3 sn t1 t2 t3 ..tm
41
Problem 6 Gene Alignment
Note, initialization, Or, Recurrence
termination Alignment start from
(n,m)-corner, Stops at (0,0) corner
42
Problem 6 Gene Alignment
43
Problem 6 Gene Alignment
? Global alignment
We may want Local alignment ?
44
Problem 6 Gene Alignment
We may want Local alignment ?
? No negative score
Initialize with 0s. Alignment, starts from
highest value on the table, And stops with a
zero
About PowerShow.com