ICS 353: Design and Analysis of Algorithms - PowerPoint PPT Presentation

About This Presentation
Title:

ICS 353: Design and Analysis of Algorithms

Description:

Divide and Conquer Reading Assignment M ... return (x,y); end if; * Analysis of Recursive MinMax Identify the divide, conquer, and combine steps in the algorithm. – PowerPoint PPT presentation

Number of Views:411
Avg rating:3.0/5.0
Slides: 28
Provided by: Was64
Category:

less

Transcript and Presenter's Notes

Title: ICS 353: Design and Analysis of Algorithms


1
ICS 353 Design and Analysis of Algorithms
King Fahd University of Petroleum
Minerals Information Computer Science Department
  • Divide and Conquer

2
Reading Assignment
  • M. Alsuwaiyel, Introduction to Algorithms Design
    Techniques and Analysis, World Scientific
    Publishing Co., Inc. 1999.
  • Chapter 6 (Except Section 6.9)

3
Divide and Conquer
  • The divide and conquer paradigm consists of the
    following steps
  • Divide step Input is partitioned into p ? 1
    partitions, each of size strictly less than n.
  • Most common value of p 2.
  • p could be equal to one when part of the input is
    discarded (example?)
  • Conquer step Perform p recursive calls if the
    problem size is greater than a specific threshold
    n0.
  • n0 is usually 1, but could be greater than 1.
  • Combine step The solution to the p recursive
    calls are combined to solve the problem for the
    union of the p partitions of the input.
  • In many, but not all cases, this step determines
    the time complexity of the algorithm.

4
Recursive Merge Sort
  • MergeSort(A,p,r)
  • if p lt r then
  • q ?(pr)/2?
  • MergeSort(A,p,q)
  • MergeSort(A,q1,r)
  • Merge(A,p,q,r)
  • end if
  • Assume that n is a power of two
  • What is the cost of MergeSort(A,j,j)?
  • What is the cost of MergeSort(A,1,n/2)?
  • What is the cost of MergeSort(A,n/21,n)?
  • What is the cost of Merge(A,1,n/2,n)?
  • What is the recurrence equation(s) describing the
    time complexity? What is the solution?

5
Recursive Binary Search Algorithm
  • BinarySearchRec(A,low,high)
  • if (low gt high) then
  • return 0
  • else
  • mid ? ?(low high)/2?
  • if x Amid then return mid
  • else if x lt Amid then
  • return BinarySearchRec(A,low,mid-1)
  • else return BinarySearchRec(A,mid1,high)
  • end if
  • end if
  • Identify the divide, conquer and combine
    operations.
  • If n 2k, what is the recurrence equation and
    its solution?

6
Recursive MinMax algorithm
  • Procedure MinMax(A,low,high)
  • if high low 1 then
  • if Alow lt Ahigh then return
    (Alow,Ahigh)
  • else return (Ahigh,Alow)
  • end if
  • else
  • mid ? ?(low high)/2?
  • (x1,y1) ? MinMax(A,low,mid)
  • (x2,y2) ? MinMax(A,mid1,high)
  • x ? minx1,x2
  • y ? maxy1,y2
  • return (x,y)
  • end if

7
Analysis of Recursive MinMax
  • Identify the divide, conquer, and combine steps
    in the algorithm.
  • Assuming n is a power of two, what is the
    recurrence equation describing the time
    complexity? What is the solution?
  • What is the cost of the straightforward
    algorithm?

8
Selection Problem
  • Problem Statement Find the kth smallest element
    in the array
  • A special case is to find the median of the array
  • In case n is odd, the median is the (n1)/2 th
    smallest element
  • In case n is even, the median is the n/2 th
    smallest element
  • What is the straightforward algorithm? What is
    its time complexity?

9
A Better Algorithm
  • If we can discard a constant fraction of the
    elements after the divide step of every recursive
    call, and recur on the rest of the elements, the
    size of the problem decreases geometrically
  • E.g. if we assume that 1/3 of the elements are
    discarded and that the algorithm spends a
    constant time per element, we get
  • cn (2/3)cn (2/3)2 cn (2/3)j cn

10
Basic Idea of the Algorithm
  • If the number of elements is less than a
    threshold, sort and find the kth element
  • Otherwise, partition the input into ?n/5? groups
    of five elements each
  • You may have a group of less than 5 elements if n
    does not divide 5.
  • Sort each group and extract its median.
  • The median of medians is computed recursively.
  • Partition the elements in A around the median
    into three sets A1, A2, and A3
  • Where to look for the kth smallest element?

11
Example
  • Find the 14th smallest element in the array
  • 8 , 33 , 17 , 51 , 57 , 49 , 35 , 11 , 25 , 37 ,
    14 , 3 , 2 , 13 , 52 , 12 , 6 , 29 , 32 , 54 , 5
    , 16 , 22 , 23 , 7 , 8 , 19 , 44 , 66

12
Algorithm Select
  • Input Array A1..n and an integer k, 1 k n
  • Output kth smallest element in A
  • Procedure select (A, low, high, k)
  • p high low 1
  • if p lt 44 then sort A and return(Ak)
  • Let q ? p/5 ?. Divide A into q groups of 5
    elements each, discarding the possibly one
    additional group with less than 5 elements
  • Sort the q groups individually extracting the
    median. Let the set of medians be M
  • mm select(M,1,q,?q/2?)
  • Partition Alow..high into three array,
    A1aaltmm, A2aamm, and A3aagtmm
  • case
  • A1 ? k return select(A1,1,A1,k)
  • A1 A2 ? k return mm
  • A1 A2 lt k return select(A3,1,A3,k(A1
    A2))

13
Analysis of the Selection Algorithm (1)
W
W
Z
Z
  • Estimating the sizes of A1 and A3.
  • A1 x ? A x ? mm
  • The size of W will give us the minimum number of
    elements in A1
  • Hence, knowing the minimum size of A1 will give
    us an upper bound on the size of which is equal
    to .

14
Analysis of the Selection Algorithm (2)
  • Now, we can write the recurrence relation for the
    selection algorithm as follows
  • What do we use to solve this recurrence?

15
Quick Sort
  • Procedure QuickSort(A,low,high)
  • if low lt high then
  • w split(A,low,high)
  • QuickSort(A,low,w-1)
  • QuickSort(A,w1,high)
  • end if

16
Split Algorithm
  • Split(A,low,high)
  • i low
  • x Alow
  • for j low 1 to high do
  • if Aj ? x then
  • i i 1
  • if i ? j then
  • swap(Ai,Aj)
  • end if
  • end if
  • end for
  • swap (Alow,Ai)
  • return i

17
Quick Sort Complexity Analysis
  • How many element comparisons are carried out by
    the split procedure?
  • Worst case analysis of Quick Sort
  • Best case analysis of Quick Sort

18
Average Case Analysis of Quick Sort
  • Assumption All permutations of the input are
    equally likely
  • The input consists of n distinct elements
  • This implies that the probability that any
    element will be picked as a pivot is
  • Let C(n) denote the number of comparisons done by
    QuickSort on the average on input of size n

19
Comparative Results of Various Sorting Algorithms
20
Multiplication of Large Integers
  • Let u and v be two n-bit integers, where n is a
    power of 2
  • The traditional multiplication algorithm takes
  • Divide and conquer can be used to carry out the
    multiplication as follows
  • Divide each integer into 2 n/2-bit
  • portions as shown here
  • u w2n/2 x and v y2n/2 z
  • The product now becomes
  • Note that multiplying a number t by 2k is
    equivalent to shifting t k bits to the left,
    which is ?(k) time.
  • How many additions and multiplications we have?
    What is the recurrence equation?

u
w
x
y
z
v
21
Can We Do Better?
  • Note that if we can reduce the number of
    multiplications by 1, we will have asymptotic
    improvement
  • Consider evaluating wz xy as follows
  • wz xy (w x) (y z) wy xz
  • Note that wy and xz have already been computed.
    So no need to compute again
  • What is the total number of multiplications now?
    Rewrite the recurrence equation and solve.

22
Matrix Multiplication
  • Let A and B be 2 n ? n matrices, assuming n to be
    a power of 2. We would like to employ divide and
    conquer to carry out the multiplication of A and
    B.
  • What is the traditional algorithm? How much does
    it cost?

23
Recursive Version
  • Let and
    then
  • What is the recurrence describing the cost of
    carrying out the multiplication by computing the
    number of multiplication operations? What is the
    solution to the recurrence? Did we achieve
    anything?

24
Strassens algorithm
  • The idea is again exactly similar to that in
    multiplying large numbers we would like to
    rewrite the product in a manner that will reduce
    the number of multiplications needed (even by
    one!)

25
Strassens Algorithm
26
Analysis of Strassens Algorithm
  • How many multiplications and additions/subtraction
    s are carried out?
  • What is the recurrence equation describing the
    cost? What is the recurrence solution?
  • Is there any improvement?

27
Empirical Comparison
n Multiplications Additions
Traditional Alg. 100 1,000,000 990,000
Strassens Alg. 100 411,822 2,470,334
Traditional Alg. 1,000 1,000,000,000 999,000,000
Strassens Alg. 1,000 264,280,285 1,579,681,709
Traditional Alg. 10,000 1012 9.99 ? 1011
Strassens Alg. 10,000 0.169 ? 1012 1012
Write a Comment
User Comments (0)
About PowerShow.com