Divide-and-Conquer

1
Divide-and-Conquer

• Recursive in structure
• Divide the problem into several smaller
  sub-problems that are similar to the original but
  smaller in size
• Conquer the sub-problems by solving them
  recursively. If they are small enough, just
  solve them in a straightforward manner.
• Combine the solutions to create a solution to the
  original problem

2
An Example Merge Sort

• Divide Divide the n-element sequence to be
  sorted into two subsequences of n/2 elements each
• Conquer Sort the two subsequences recursively
  using merge sort.
• Combine Merge the two sorted subsequences to
  produce the sorted answer.

3
Merge-Sort (A, p, r)

• INPUT a sequence of n numbers stored in array A
• OUTPUT an ordered sequence of n numbers
• 1. if p lt r
• 2. then q ? (pr)/2
• 3. Merge-Sort (A, p, q)
• 4. Merge-Sort (A, q1, r)
• 5. Merge (A, p, q, r)

4
Analysis of Merge Sort

• Divide computing the middle takes ? (1)
• Conquer solving 2 sub-problem takes 2T(n/2)
• Combine merging n-element takes ? (n)
• Total
• T(n) ? (1) if n 1
• T(n) 2T(n/2) ? (n) if n gt 1
• ? T(n) ? (n lg n) (CLRS/Chapter 4)

5
Recurrence Relations

• Recurrences (Chapter 4)
• Substitution Method
• Iteration Method
• Master Method
• Arising from Divide and Conquer
• (e.g. MERGE-SORT)
• T(n) ? (1) if n ? c
• T(n) a T(n/b) D(n) C(n) otherwise

6
Substitution Method

• Guessing the form of the solutions, then using
  mathematical induction to find the constants and
  show the solution works.
• It works well when it is easy to guess. But,
  there is no general way to guess the correct
  solution.

7
An Example

• Solve T(n) 3T(?n/3?) n
• T(n) ? 3c ?n/3? lg ?n/3? n
• ? c n lg (n/3) n
• c n lg n - c n lg3 n
• c n lg n - n (c lg 3 - 1)
• ? c n lg n
• The last step is true for c ? 1 / lg3.

8
Making a Good Guess

• Guessing a similar solution to the one that you
  have seen before
• T(n) 3T(?n/3? 5) n similar to T(n)
  3T(?n/3?) n when n is large, the difference
  between n/3 and (n/3 5) is insignificant
• Another way is to prove loose upper and lower
  bounds on recurrence and then reduce the range of
  uncertainty.
• Start with T(n) ?(n) T(n) O(n2) ? T(n) ?
  (n log n)

9
Subtleties

• When the math doesnt quite work out in the
  induction, try to adjust your guess with a
  lower-order term. For example
• We guess T(n) ? O(n) for T(n) 3T(?n/3?) 4, but
  we have T(n) ? 3c ?n/3? 4 c n 4
• New guess is T(n) ? c n - b, where b ? 0
• T(n) ? 3(c ?n/3? - b)4 c n - 3b 4 c n
  - b - (2b-4)
• Therefore, T(n) ? c n - b, if 2b - 4 ? 0
  or if b ? 2

10
Changing Variables

• Use algebraic manipulation to turn an unknown
  recurrence similar to what you have seen before.
  
• Consider T(n) 2T(?n1/2?) lg n
• Rename m lg n and we have
• T(2m) 2T(2m/2) m
• Set S(m) T(2m) and we have
• S(m) 2S(m/2) m ? S(m) O(m lg m)
• Changing back from S(m) to T(n), we have
• T(n) T(2m) S(m) O(m lg m) O(lg n lg
  lg n)

11
Avoiding Pitfalls

• Be careful not to misuse asymptotic notation.
  For example
• We can falsely prove T(n) O(n) by guessing T(n)
  ? c n for T(n) 2T(?n/2?) n
• T(n) ? 2c ?n/2? n
• ? c n n
• O(n) ? Wrong!
• The err is that we havent proved T(n) ? c n

12
Exercises

• Solution of T(n) T(?n/2?) 1 is O(lg n)
• Solution of T(n) 2T(?n/2? 17) n is O(n lg
  n)
• Solve T(n) 2T(n1/2) 1 by making a change of
  variables. Dont worry whether values are
  integral.

13
Iteration Method

• Expand (iterate) the recurrence and express it as
  a summation of terms dependent only on n and the
  initial conditions
• The key is to focus on 2 parameters
• the number of times the recurrence needs to be
  iterated to reach the boundary condition
• the sum of terms arising from each level of the
  iteration process
• Techniques for evaluating summations can then be
  used to provide bounds on solution.

14
An Example

• Solve T(n) 3T(n/4) n
• T(n) n 3T(n/4)
• n 3 n/4 3T(n/16)
• n 3n/4 9T(n/16)
• n 3n/4 9 n/16 27T(n/64)
• T(n) ? n 3n/4 9n/16 27n/64 3log4
  n?(1)
• ? n ? (3/4)i ?(nlog43)
• 4n o(n)
• O(n)

15
Recursion Trees

• Keep track of the time spent on the subproblems
  of a divide and conquer algorithm
• A convenient way to visualize what happens when a
  recursion is iterated
• Help organize the algebraic bookkeeping necessary
  to solve the recurrence

16
Merge Sort
Running times to merge two sublists
n
Running time to sort the left sublist
17
Running Time
nn
n
2(n/2) n
n/2
n/2
lg n
4(n/4) n
n/4
n/4
n/4
n/4








Total n lg n
18
Recursion Trees and Recurrences

• Useful even when a specific algorithm is not
  specified
• For T(n) 2T(n/2) n2, we have

19
Recursion Trees

• T(n) ?(n2)

20
Recursion Trees

• For T(n) T(n/3) T(2n/3) n
• T(n) O(n lg n)

21
Master Method

• Provides a cookbook method for solving
  recurrences of the form T(n) a T(n/b) f(n)
• Assumptions
• a ? 1 and b ? 1 are constants
• f(n) is an asymptotically positive function
• T(n) is defined for nonnegative integers
• We interpret n/b to mean either ?n/b? or ?n/b?

22
The Master Theorem

• With the recurrence T(n) a T(n/b) f(n) as in
  the previous slide, T(n) can be bounded
  asymptotically as follows
• 1. If f(n)O(nlogba-?) for some constant ? gt 0,
  then T(n) ?(nlogba).
• 2. If f(n) ?(nlogba), then T(n) ?(nlogba lg
  n).
• 3. If f(n) ? ( nlogba? ) for some constant ?
  gt 0, and if a f(n/b) ? c f(n) for some constant c
  lt 1 and all sufficiently large n, then T(n)
  ?(f(n)).

23
Simplified Master Theorem

• Let a ? 1 and b gt 1 be constants and let T(n) be
  the recurrence
• T(n) a T(n/b) c nk
• defined for n ? 0.
• 1. If a gt bk, then T(n) ?( nlogba ).
• 2. If a bk, then T(n) ?( nk lg n ).
• 3. If a lt bk, then T(n) ?( nk ).

24
Examples

• T(n) 16T(n/4) n
• a 16, b 4, thus nlogba nlog416 ?(n2)
• f(n) n O(nlog416 - ? ) where ? 1 ? case
  1.
• Therefore, T(n) ?(nlogba ) ?(n2)
• T(n) T(3n/7) 1
• a 1, b7/3, and nlogba nlog 7/3 1 n0 1
• f(n) 1 ?(nlogba) ? case 2.
• Therefore, T(n) ?(nlogba lg n) ?(lg n)

25
Examples (Cont.)

• T(n) 3T(n/4) n lg n
• a 3, b4, thus nlogba nlog43 O(n0.793)
• f(n) n lg n ?(nlog43 ? ) where ? ? 0.2 ?
  case 3.
• Therefore, T(n) ?(f(n)) ?(n lg n)
• T(n) 2T(n/2) n lg n
• a 2, b2, f(n) n lg n, and nlogba nlog22
  n
• f(n) is asymptotically larger than nlogba, but
  not polynomially larger. The ratio lg n is
  asymptotically less than n? for any positive ?.
  Thus, the Master Theorem doesnt apply here.

26
Exercises

• Use the Master Method to solve the following
• T(n) 4T(n/2) n
• T(n) 4T(n/2) n2
• T(n) 4T(n/2) n3