CS 3343: Analysis of Algorithms - PowerPoint PPT Presentation

About This Presentation
Title:

CS 3343: Analysis of Algorithms

Description:

CS 3343: Analysis of Algorithms Lecture 6&7: Master theorem and substitution method Analyzing recursive algorithms Defining recurrence Solving recurrence Solving ... – PowerPoint PPT presentation

Number of Views:155
Avg rating:3.0/5.0
Slides: 39
Provided by: DavidLu155
Learn more at: http://www.cs.utsa.edu
Category:

less

Transcript and Presenter's Notes

Title: CS 3343: Analysis of Algorithms


1
CS 3343 Analysis of Algorithms
  • Lecture 67 Master theorem and substitution
    method

2
  • Analyzing recursive algorithms
  • Defining recurrence
  • Solving recurrence

3
Solving recurrence
  • Recursion tree / iteration method
  • - Good for guessing an answer
  • Substitution method
  • - Generic method, rigid, but may be hard
  • Master method
  • - Easy to learn, useful in limited cases only
  • - Some tricks may help in other cases

4
The master method
The master method applies to recurrences of the
form T(n) a T(n/b) f (n) , where a ³ 1, b gt
1, and f is asymptotically positive.
  • Divide the problem into a subproblems, each of
    size n/b
  • Conquer the subproblems by solving them
    recursively.
  • Combine subproblem solutions
  • Divide combine takes f(n) time.

5
Master theorem
T(n) a T(n/b) f (n)
Key compare f(n) with nlogba
CASE 1 f (n) O(nlogba e) ? T(n) Q(nlogba)
. CASE 2 f (n) Q(nlogba) ? T(n) Q(nlogba
log n) . CASE 3 f (n) W(nlogba e) and a f
(n/b) c f (n) ? T(n) Q( f (n)) .
Regularity Condition
6
Case 1
f (n) O(nlogba e) for some constant e gt
0. Alternatively nlogba / f(n)
O(ne) Intuition f (n) grows polynomially slower
than nlogba Or nlogba dominates f(n) by an ne
factor for some e gt 0 Solution T(n) Q(nlogba)
T(n) 4T(n/2) n b 2, a 4, f(n) n log24
2 f(n) n O(n2-e), or n2 / n n1 O(n?),
for e 1 ? T(n) T(n2)
T(n) 2T(n/2) n/logn b 2, a 2, f(n) n /
log n log22 1 f(n) n/logn ? O(n1-e), or n1/
f(n) log n ? O(ne), for any e gt 0 ? CASE 1 does
not apply
7
Case 2
f (n) Q (nlogba). Intuition f (n) and nlogba
have the same asymptotic order. Solution T(n)
Q(nlogba log n)
e.g. T(n) T(n/2) 1 logba 0 T(n) 2
T(n/2) n logba 1 T(n) 4T(n/2) n2
logba 2 T(n) 8T(n/2) n3 logba 3
8
Case 3
f (n) O(nlogba ?) for some constant ? gt
0. Alternatively f(n) / nlogba
O(n?) Intuition f (n) grows polynomially faster
than nlogba Or f(n) dominates nlogba by an n?
factor for some ? gt 0 Solution T(n) T(f(n))
T(n) T(n/2) n b 2, a 1, f(n)
n nlog21 n0 1 f(n) n O(n0?), or n / 1 n
O(n?) ? T(n) T(n)
T(n) T(n/2) log n b 2, a 1, f(n) log
n nlog21 n0 1 f(n) log n ? O(n0?),
or f(n) / nlog21 / log n ? O(ne) ? CASE 3 does
not apply
9
Regularity condition
  • a f (n/b) c f (n) for some c lt 1 and all
    sufficiently large n
  • This is needed for the master method to be
    mathematically correct.
  • to deal with some non-converging functions such
    as sine or cosine functions
  • For most f(n) youll see (e.g., polynomial,
    logarithm, exponential), you can safely ignore
    this condition, because it is implied by the
    first condition f (n) O(nlogba ?)

10
Examples
T(n) 4T(n/2) n a 4, b 2 ? nlogba n2 f
(n) n. CASE 1 f (n) O(n2 e) for e 1. ?
T(n) Q(n2).
T(n) 4T(n/2) n2 a 4, b 2 ? nlogba n2
f (n) n2. CASE 2 f (n) Q(n2). ? T(n)
Q(n2log n).
11
Examples
T(n) 4T(n/2) n3 a 4, b 2 ? nlogba n2
f (n) n3. CASE 3 f (n) W(n2 e) for e
1 and 4(n/2)3 cn3 (reg. cond.) for c 1/2. ?
T(n) Q(n3).
T(n) 4T(n/2) n2/log n a 4, b 2 ? nlogba
n2 f (n) n2/log n. Master method does not
apply. In particular, for every constant e gt 0,
we have ne w(log n).
12
Examples
T(n) 4T(n/2) n2.5 a 4, b 2 ? nlogba
n2 f (n) n2.5. CASE 3 f (n) W(n2 e) for
e 0.5 and 4(n/2)2.5 cn2.5 (reg. cond.) for c
0.75. ? T(n) Q(n2.5).
T(n) 4T(n/2) n2 log n a 4, b 2 ? nlogba
n2 f (n) n2log n. Master method does not
apply. In particular, for every constant e gt 0,
we have ne w(log n).
13
  • How do I know which case to use? Do I need to try
    all three cases one by one?

14
  • Compare f(n) with nlogba
  • o(nlogba) Possible CASE 1
  • f(n) ? T(nlogba) CASE 2
  • ?(nlogba) Possible CASE 3

check if nlogba / f(n) ? O(n?)
check if f(n) / nlogba ? O(n?)
15
Examples
logba 2. n o(n2) gt Check case 1
logba 2. n2 o(n2) gt case 2
logba 1.3. n o(n1.3) gt Check case 1
logba 0.5. n ?(n0.5) gt Check case 3
logba 0. nlogn ?(n0) gt Check case 3
logba 1. nlogn ?(n) gt Check case 3
16
More examples
17
Some tricks
  • Changing variables
  • Obtaining upper and lower bounds
  • Make a guess based on the bounds
  • Prove using the substitution method

18
Changing variables
T(n) 2T(n-1) 1
  • Let n log m, i.e., m 2n
  • gt T(log m) 2 T(log (m/2)) 1
  • Let S(m) T(log m) T(n)
  • gt S(m) 2S(m/2) 1
  • gt S(m) T(m)
  • gt T(n) S(m) T(m) T(2n)

19
Changing variables
  • Let n 2m
  • gt sqrt(n) 2m/2
  • We then have T(2m) T(2m/2) 1
  • Let T(n) T(2m) S(m)
  • gt S(m) S(m/2) 1
  • S(m) T (log m) T (log log n)
  • T(n) T (log log n)

20
Changing variables
  • T(n) 2T(n-2) 1
  • Let n log m, i.e., m 2n
  • gt T(log m) 2 T(log m/4) 1
  • Let S(m) T(log m) T(n)
  • gt S(m) 2S(m/4) 1
  • gt S(m) m1/2
  • gt T(n) S(m) (2n)1/2 (sqrt(2)) n ? 1.4n

21
Obtaining bounds
  • Solve the Fibonacci sequence
  • T(n) T(n-1) T(n-2) 1
  • T(n) gt 2T(n-2) 1 1
  • T(n) lt 2T(n-1) 1 2
  • Solving 1, we obtain T(n) gt 1.4n
  • Solving 2, we obtain T(n) lt 2n
  • Actually, T(n) ? 1.62n

22
Obtaining bounds
  • T(n) T(n/2) log n
  • T(n) ? O(log n)
  • T(n) ? O(T(n/2) n?)
  • Solving T(n) T(n/2) n?,
  • we obtain T(n) O(n?), for any ? gt 0
  • So T(n) ?O(n?) for any ? gt 0
  • T(n) is unlikely polynomial
  • Actually, T(n) T(log2n) by extended case 2

23
Extended Case 2
  • CASE 2 f (n) Q(nlogba) ? T(n) Q(nlogba log
    n).
  • Extended CASE 2 (k gt 0)
  • f (n) Q(nlogba logkn) ? T(n) Q(nlogba
    logk1n).

24
Solving recurrence
  • Recursion tree / iteration method
  • - Good for guessing an answer
  • - Need to verify
  • Substitution method
  • - Generic method, rigid, but may be hard
  • Master method
  • - Easy to learn, useful in limited cases only
  • - Some tricks may help in other cases

25
Substitution method
The most general method to solve a recurrence
(prove O and ? separately)
  • Guess the form of the solution
  • (e.g. by recursion tree / iteration method)
  • Verify by induction (inductive step).
  • Solve for O-constants n0 and c (base case of
    induction)

26
Proof by substitution
  • Recurrence T(n) 2T(n/2) n.
  • Guess T(n) O(n log n). (eg. by recursion tree
    method)
  • To prove, have to show T(n) c n log n for
    some c gt 0 and for all n gt n0
  • Proof by induction assume it is true for T(n/2),
    prove that it is also true for T(n). This means
  • Given T(n) 2T(n/2) n
  • Need to Prove T(n) c n log (n)
  • Assume T(n/2) cn/2 log (n/2)

27
Proof
  • Given T(n) 2T(n/2) n
  • Need to Prove T(n) c n log (n)
  • Assume T(n/2) cn/2 log (n/2)
  • Proof
  • Substituting T(n/2) cn/2 log (n/2) into the
    recurrence, we get
  • T(n) 2 T(n/2) n
  • cn log (n/2) n
  • c n log n - c n n
  • c n log n - (c - 1) n
  • c n log n for all n gt 0 (if c 1).
  • Therefore, by definition, T(n) O(n log n).

28
Proof by substitution
  • Recurrence T(n) 2T(n/2) n.
  • Guess T(n) O(n log n).
  • To prove, have to show T(n) c n log n for
    some c gt 0 and for all n gt n0
  • Proof by induction assume it is true for T(n/2),
    prove that it is also true for T(n). This means
  • Given
  • Need to Prove T(n) c n log (n)
  • Assume

T(n) 2T(n/2) n
T(n/2) cn/2 log (n/2)
29
Proof
  • Given T(n) 2T(n/2) n
  • Need to Prove T(n) c n log (n)
  • Assume T(n/2) cn/2 log (n/2)
  • Proof
  • Substituting T(n/2) cn/2 log (n/2) into the
    recurrence, we get
  • T(n) 2 T(n/2) n
  • cn log (n/2) n
  • c n log n - c n n
  • c n log n (1 c) n
  • c n log n for all n gt 0 (if c 1).
  • Therefore, by definition, T(n) O(n log n).

30
More substitution method examples (1)
  • Prove that T(n) 3T(n/3) n O(nlogn)
  • Need to prove that T(n) ? c n log n for some c,
    and sufficiently large n
  • Assume above is true for T(n/3), i.e.
  • T(n/3) ? cn/3 log (n/3)

31
  • T(n) 3 T(n/3) n
  • ? 3 cn/3 log (n/3) n
  • ? cn log n cn log3 n
  • ? cn log n (cn log3 n)
  • ? cn log n (if cn log3 n 0)
  • cn log3 n 0
  • gt c log 3 1 0 (for n gt 0)
  • gt c 1/log3
  • gt c log32
  • Therefore, T(n) 3 T(n/3) n ? cn log n for c
    log32 and n gt 0. By definition, T(n) O(n log n).

32
More substitution method examples (2)
  • Prove that T(n) T(n/3) T(2n/3) n O(nlogn)
  • Need to prove that T(n) ? c n log n for some c,
    and sufficiently large n
  • Assume above is true for T(n/3) and T(2n/3), i.e.
  • T(n/3) ? cn/3 log (n/3)
  • T(2n/3) ? 2cn/3 log (2n/3)

33
  • T(n) T(n/3) T(2n/3) n
  • ? cn/3 log(n/3) 2cn/3 log(2n/3) n
  • ? cn log n n cn (log 3 2/3)
  • ? cn log n n(1 clog3 2c/3)
  • ? cn log n, for all n gt 0 (if 1 c log3 2c/3
    ? 0)
  • c log3 2c/3 1
  • c 1 / (log3-2/3) gt 0
  • Therefore, T(n) T(n/3) T(2n/3) n ? cn log n
    for c 1 / (log3-2/3) and n gt 0. By definition,
    T(n) O(n log n).

34
More substitution method examples (3)
  • Prove that T(n) 3T(n/4) n2 O(n2)
  • Need to prove that T(n) ? c n2 for some c, and
    sufficiently large n
  • Assume above is true for T(n/4), i.e.
  • T(n/4) ? c(n/4)2 cn2/16

35
  • T(n) 3T(n/4) n2
  • ? 3 c n2 / 16 n2
  • ? (3c/16 1) n2
  • ? cn2
  • 3c/16 1 ? c implies that c 16/13
  • Therefore, T(n) 3(n/4) n2 ? cn2 for c 16/13
    and all n. By definition, T(n) O(n2).

?
36
Avoiding pitfalls
  • Guess T(n) 2T(n/2) n O(n)
  • Need to prove that T(n) ? c n
  • Assume T(n/2) ? cn/2
  • T(n) ? 2 cn/2 n cn n O(n)
  • Whats wrong?
  • Need to prove T(n) ? cn, not T(n) ? cn n

37
Subtleties
  • Prove that T(n) T(?n/2?) T(?n/2?) 1 O(n)
  • Need to prove that T(n) ? cn
  • Assume above is true for T(?n/2?) T(?n/2?)
  • T(n) lt c ?n/2? c?n/2? 1
  • ? cn 1
  • Is it a correct proof?
  • No! has to prove T(n) lt cn
  • However we can prove T(n) O (n 1)

38
Making good guess
  • T(n) 2T(n/2 17) n
  • When n approaches infinity, n/2 17 are not too
    different from n/2
  • Therefore can guess T(n) ?(n log n)
  • Prove ?
  • Assume T(n/2 17) c (n/217) log (n/2 17)
  • Then we have
  • T(n) n 2T(n/217)
  • n 2c (n/217) log (n/2 17)
  • n c n log (n/2 17) 34 c log (n/217)
  • c n log (n/2 17) 34 c log (n/217)
  • .
  • Maybe can guess T(n) ?((n-17) log (n-17))
    (trying to get rid of the 17).
  • Details skipped.
Write a Comment
User Comments (0)
About PowerShow.com