Unit 1: Algorithmic Fundamentals - PowerPoint PPT Presentation

Loading...

PPT – Unit 1: Algorithmic Fundamentals PowerPoint presentation | free to view - id: 77c8d1-MGRkM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Unit 1: Algorithmic Fundamentals

Description:

Course contents: On algorithms Mathematical foundations Asymptotic notation Growth of functions Complexity Lower vs. upper bounds Recurrences Unit 1: Algorithmic ... – PowerPoint PPT presentation

Number of Views:73
Avg rating:3.0/5.0
Slides: 86
Provided by: edut1550
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Unit 1: Algorithmic Fundamentals


1
Unit 1 Algorithmic Fundamentals
  • Course contents
  • On algorithms
  • Mathematical foundations
  • Asymptotic notation
  • Growth of functions
  • Complexity
  • Lower vs. upper bounds
  • Recurrences

2
On Algorithms
  • Algorithm A well-defined procedure for
    transforming some input to a desired output.
  • Major concerns
  • Correctness Does it halt? Is it correct? Is it
    stable?
  • Efficiency Time complexity? Space complexity?
  • Worst case? Average case? (Best case?)
  • Better algorithms?
  • How Faster algorithms? Algorithms with less
    space requirement?
  • Optimality Prove that an algorithm is best
    possible/optimal? Establish a lower bound?

3
Example Traveling Salesman Problem (TSP)
  • Input A set of points (cities) P together with a
    distance d(p, q) between any pair p, q ? P.
  • Output What is the shortest circular route that
    starts and ends at a given point and visits all
    the points.
  • Correct and efficient algorithms?

4
Nearest Neighbor Tour
1. pick and visit an initial point p0 2. P ? p0
3. i ? 0 4. while there are unvisited points
do 5. visit pi's closet unvisited point
pi1 6. i ? i 1 7. return to p0 from pi.
  • Simple to implement and very efficient, but
    incorrect!

5
A Correct, but Inefficient Algorithm
1. d ? ? 2. for each of the n! permutations ?i
of the n points 3. if (cost(?i) ? d) then
4. d ? cost(?i) 5. Tmin ?
?i 6. return Tmin.
  • Correctness Tries all possible orderings of the
    points ? Guarantees to end up with the shortest
    possible tour.
  • Efficiency Tries n! possible routes!
  • 120 routes for 5 points, 3,628,800 routes for 10
    points, 20 points?
  • No known efficient, correct algorithm for TSP!
  • TSP is NP-complete.

6
Example Sorting
  • Input A sequence of n numbers lta1, a2, , angt.
  • Output A permutation lta1', a2', , an'gt such
    that a1' ? a2' ? ? an'.
  • Input lt8, 6, 9, 7, 5, 2, 3gt
  • Output lt2, 3, 5, 6, 7, 8, 9 gt
  • Correct and efficient algorithms?

7
Insertion Sort
InsertionSort(A) 1. for j ? 2 to lengthA do 2.
key ? Aj 3. / Insert Aj into the
sorted sequence A1..j-1. / 4. i ? j - 1
5. while i gt 0 and Ai gt key do 6.
Ai1 ? Ai 7. i ? i - 1 8.
Ai1 ? key
8
Exact Analysis of Insertion Sort
  • The for loop is executed (n-1) 1 times. (why?)
  • tj of times the while loop test for value j
    (i.e., 1 of elements that have to be slided
    right to insert the j-th item).
  • Step 5 is executed t2 t3 tn times.
  • Step 6 is executed (t2 - 1) (t3 - 1) (tn
    - 1) times.

9
Exact Analysis of Insertion Sort (contd)
  • Best case If the input is already sorted, all
    tj's are 1.
  • Linear T(n) (c1 c2 c4 c5 c8)n -
    (c2 c4 c5 c8)
  • Worst case If the array is in reverse sorted
    order, tj j, ? j.
  • Quadratic T(n) (c5 /2 c6/ 2 c7/2 ) n2
    (c1 c2 c4 c5 /2 c6 /2 c7/2 c8) n
    (c2 c4 c5 c8)
  • Exact analysis is often hard!

10
Asymptotic Analysis
  • Asymptotic analysis looks at growth of T(n) as n
    ? ?.
  • ? notation Drop low-order terms and ignore the
    leading constant.
  • E.g., 8n3 - 4n2 5n - 2 ?(n3).
  • As n grows large, lower-order ? algorithms
    outperform higher-order ones.
  • Worst case input reverse sorted, while loop is
    ?(j)
  • Average case all permutations equally likely,
    while loop is ?(j / 2)

11
Merge Sort A Divide-and-Conquer Algorithm
MergeSort(A, p, r)
T(n) 1. If p lt r then
?(1) 2. q ? ? (pr)/2?
?(1) 3.
MergeSort (A, p, q)
T(n/2) 4. MergeSort (A, q 1, r)
T(n/2) 5. Merge(A, p, q, r)
?(n)
12
Recurrence
  • Describes a function recursively in terms of
    itself.
  • Describes performance of recursive algorithms.
  • Recurrence for merge sort

MergeSort(A, p, r)
T(n) 1. If p lt r then
?(1) 2. q ? ? (pr)/2?
?(1) 3. MergeSort (A, p,
q) T(n/2) 4. MergeSort
(A, q 1, r) T(n/2) 5.
Merge(A, p, q, r) ?(n)
13
Recursion Tree for Asympotatic Analysis
  • ?(n lg n) grows more slowly than ?(n2).
  • Thus merge sort asymptotically beats insertion
    sort in the worst case. (insertion sort
    stable/in-place merge sort stable/not in-place)

14
O Upper Bounding Function
  • Def f(n) O(g(n)) if ? c gt0 and n0 gt 0 such that
    0 ? f(n) ? cg(n) for all n ? n0.
  • Intuition f(n) ? g(n) when we ignore constant
    multiples and small values of n.
  • How to show O (Big-Oh) relationships?
  • f(n) O(g(n)) iff limn ? ? c for some
    c ? 0.
  • Remember L'Hopitals Rule?

15
? Lower Bounding Function
  • Def f(n) ?(g(n)) if ? c gt0 and n0 gt 0 such that
    0 ? cg(n) ? f(n) for all n ? n0.
  • Intuition f(n) ? g(n) when we ignore
    constant multiples and small values of n.
  • How to show ? (Big-Omega) relationships?
  • f(n) ?(g(n)) iff limn ? ? c for some
    c ? 0.

16
? Tightly Bounding Function
  • Def f(n) ?(g(n)) if ? c1, c2 gt0 and n0 gt 0 such
    that 0 ? c1g(n) ? f(n) ? c2 g(n) for all n ? n0.
  • Intuition f(n) g(n) when we ignore
    constant multiples and small values of n.
  • How to show ? relationships?
  • Show both big Oh (O) and Big Omega (?)
    relationships.
  • f(n) ?(g(n)) iff limn ? ? c for some
    c gt 0.

17
o, ? Untightly Upper, Lower Bounding Functions
  • Little Oh o f(n) o(g(n)) if ? c gt 0, ? n0 gt 0
    such that 0 ? f(n) lt cg(n) for all n ? n0.
  • Intuition f(n) lt any constant multiple of
    g(n) when we ignore small values of n.
  • Little Omega ? f(n) ?(g(n)) if ? c gt 0, ? n0 gt
    0 such that 0 ? cg(n) lt f(n) for all n ? n0.
  • Intuition f(n) is gt any constant multiple of
    g(n) when we ignore small values of n.
  • How to show o (Little-Oh) and ?(Little-Omega)
    relationships?
  • f(n) o(g(n)) iff limn ? ? 0.
  • f(n) ?(g(n)) iff limn ? ? ?.

18
Properties for Asymptotic Analysis
  • An algorithm has worst-case run time O(f(n))
    there is a constant c s.t. for every n big
    enough, every execution on an input of size n
    takes at most cf(n) time.
  • An algorithm has worst-case run time ?(f(n))
    there is a constant c s.t. for every n big
    enough, at least one execution on an input of
    size n takes at least cf(n) time.
  • Transitivity If f(n) ?(g(n)) and g(n)
    ?(h(n)), then f(n) ?(h(n)), where ? O, o, ?,
    ?, or ?.
  • Rule of sums ?(f(n) g(n)) ?(maxf(n),
    g(n)), where ? O, o, ?, ?, or ?.
  • Rule of sums f(n) g(n) ?(maxf(n), g(n)),
    where ? O, ?, or ?.
  • Rule of products If f1(n) ?(g1(n)) and f2(n)
    ?(g2(n)), then f1(n) f2(n) ?(g1(n) g2(n)),
    where ? O, o, ?, ?, or ?.
  • Transpose symmetry f(n) O(g(n)) iff g(n)
    ?(f(n)).
  • Transpose symmetry f(n) o(g(n)) iff g(n)
    ?(f(n)).
  • Reflexivity f(n) ?(f(n)), where ? O, ? , or
    ?.
  • Symmetry f(n) ?(g(n)) iff g(n) ?(f(n)).

19
Asymptotic Functions
  • Polynomial-time complexity O(p(n)), where n is
    the input size and p(n) is a polynomial function
    of n (p(n) nO(1)).

20
Runtime Comparison
  • Run-time comparison Assume 1000 MIPS, 1
    instruction/operation.

21
Cant Finish the Assigned Task
I cant find an efficient algorithm, I guess Im
just too dumb.
22
Mission Impossible
I cant find an efficient algorithm, because no
such algorithm is possible.
23
I cant find an efficient algorithm, but neither
can all these famous people.
24
Easy and Hard Problems
  • We argue that the class of problems that can be
    solved in polynomial time (denoted by P)
    corresponds well with what we can feasibly
    compute. But sometimes it is difficult to tell
    when a particular problem is in P or not.
  • Theoreticians spend a good deal of time trying to
    determine whether particular problems are in P.
    To demonstrate how difficult it can be.
  • To make this determination, we will survey a
    number of problems, some of which are known to be
    in P, and some of which we think are (probably)
    not in P. The difference between the two types
    of problem can be surprisingly small. Throughout
    the following, an ''easy'' problem is one that is
    solvable in polynomial time, while a ''hard''
    problem is one that we think cannot be solved in
    polynomial time.

25
Eulerian Tour vs. Hamiltonian Tour
-- Easy
  • Eulerian Tours
  • INPUT A graph G (V, E).
  • DECIDE Is there a path that crosses every edge
    exactly once and returns to its starting point?
  • Hamiltonian Tours
  • INPUT A graph G (V, E).
  • DECIDE Is there a path that visits every vertex
    exactly once and returns to its starting point?

-- Hard
26
Some Facts
  • Eulerian Tours
  • A famous mathematical theorem comes to our
    rescue. If the graph is connected and every
    vertex has even degree, then the graph is
    guaranteed to have such a tour. The algorithm to
    find the tour is a little trickier, but still
    doable in polynomial time.
  • Hamiltonian Tours
  • No one knows how to solve this problem in
    polynomial time. The subtle distinction between
    visiting edges and visiting vertices changes an
    easy problem into a hard one.

27
Map Colorability
-- Easy
  • Map 2-colorability
  • INPUT A graph G(V, E).
  • DECIDE Can this map be
  • colored with 2 colors so that no
  • two adjacent countries have the
  • same color?
  • Map 3-colorability
  • INPUT A graph G(V, E).
  • DECIDE Can this map be colored with 3 colors so
    that no two adjacent countries have the same
    color?
  • Map 4-colorability

-- Hard
-- Easy
28
Some Facts
  • Map 2-colorability
  • To solve this problem, we simply color the first
    country arbitrarily. This forces the colors of
    neighboring countries to be the other color,
    which in turn forces the color of the countries
    neighboring those countries, and so on. If we
    reach a country which borders two countries of
    different color, we will know that the map cannot
    be two-colored otherwise, we will produce a two
    coloring. So this problem is easily solvable in
    polynomial time.

29
Some Facts
  • Map 3-colorability
  • This problem seems very similar to the problem
    above, however, it turns out to be much harder.
    No one knows how this problem can be solved in
    polynomial time. (In fact this problem is
    NP-complete.)
  • Map 4-colorability.
  • Here we have an easy problem again. By a famous
    theorem, any map can be four-colored. It turns
    out that finding such a coloring is not that
    difficult either.

30
Problem vs. Problem Instance
  • When we say that a problem is hard, it means that
    some instances of the problem are hard. It does
    not mean that all problem instances are hard.
  • For example, the following problem instance is
    trivially 3-colorable

31
Longest Path vs. Shortest Path
-- Hard
  • Longest Path
  • INPUT A graph G (V, E), two vertices u, v of
    V, and a weighting function on E.
  • OUTPUT The longest path between u and v.
  • Shortest Path
  • INPUT A graph G (V, E), two vertices u, v of
    V, and a weighting function on E.
  • OUTPUT The shortest path between u and v.

No one is able to come up with a polynomial time
algorithm yet.
-- Easy
A greedy method will solve this problem easily.
32
Multiplication vs. Factoring
-- Easy
  • Multiplication
  • INPUT Integers x,y.
  • OUTPUT The product x?y.
  • Factoring (Un-multiplying)
  • INPUT An integer n.
  • OUTPUT If n is not prime, output two integers x,
    y such that 1 lt x, y lt n and x ? y n.

-- Hard
Again, the problem of factoring is not known to
be in P. In this case, the hardness of a problem
turns out to be useful. Some cryptographic
algorithms depend on the assumption that
factoring is hard to ensure that a code cannot be
broken by a computer.
33
Boolean Formulas
  • Formula evaluation
  • INPUT A boolean formula (e.g. (x ?y) ? (z ??x))
    and a value for all variables in the formula
    (e.g. x 0, y 1, z 0).
  • DECIDE The value of the formula. (e.g., 1, or
    "true'' in this case).
  • Satisfiability of boolean formula
  • INPUT A boolean formula.
  • DECIDE Do there exist values for all variables
    that would make the formula true?
  • Tautology
  • INPUT A boolean formula.
  • DECIDE Do all possible assignments of values to
    variables make the formula true?

-- Easy
-- Hard
-- Harder
34
Facts
  • Formula evaluation
  • It's not too hard to think of what the algorithm
    would be in this case. All we would have to do is
    to substitute the values in for the various
    variables, then simplify the formula to a single
    value in multiple passes (e.g. in a pass simplify
    1 ? 0 to 1). .
  • Satisfiability of boolean formula
  • Given that there are n different variables in the
    formula, there are 2n possible assignments of 0/1
    to the variables. This gives us an easy
    exponential time algorithm simply try all
    possible assignments. No one knows if there is a
    way to be cleverer, and cut the running time down
    to polynomial
  • Tautology
  • It turns out that this problem seems to be even
    harder than the Satisfiability problem.

35
How Do You Judge an Algorithm?
  • Issues Related to the analysis of Algorithms
  • How to measure the goodness of an algorithm?
  • How to measure the difficulty of a problem?
  • How do we know that an algorithm is optimal?

36
The Complexity of an Algorithm
  • The space complexity of a program is the amount
    of memory that it needs to run to completion.
  • Fixed space requirements does not depend on the
    programs inputs and outputs -- usually ignored.
  • Variable space requirement size depends on
    execution of program (recursion, dynamic
    allocated variables, etc.)
  • The time complexity of a program is the amount of
    computer time that it needs to run a computation.

37
Input (Problem) Size
  • Input (problem) size and costs of operations The
    size of an instance corresponds formally to the
    number of bits needed to represent the instance
    on a computer, using some precisely defined and
    reasonably compact coding scheme.
  • uniform cost function
  • logarithmic cost function
  • Example Compute x nn
  • x 1 uniform logarithmic
  • for i 1 to n do T(n) Q(n) T(n)
    Q(n2?log n)
  • x x n S(n) Q(1) S(n) Q(n ?
    log n)

38
Complexity of an Algorithm
  • Best case analysis too optimistic, not really
    useful.
  • Worst case analysis usually only yield a rough
    upper bound.
  • Average case analysis a probability distribution
    of input is assumed, and the average of the cost
    of all possible input patterns are calculated.
    However, it is usually difficult than worst case
    analysis and does not reflect the behavior of
    some specific data patterns.
  • Amortized analysis this is similar to average
    case analysis except that no probability
    distribution is assumed and it is applicable to
    any input pattern (worst case result).
  • Competitive analysis Used to measure the
    performance of an on-line algorithm w.r.t. an
    adversary or an optimal off-line algorithm.

39
Example Binary Search
  • Given a sorted array A1..n and an item x in A.
    What is the index of x in A?
  • Usually, the best case analysis is the easiest,
    the worst case the second easiest, and the
    average analysis the hardest.

40
Another Example
  • Given a stack S with 2 operations push(S, x),
    and multipop(S, k), the cost of the two
    operations are 1 and min(k, S) respectively.
    What is the cost of a sequence of n operations on
    an initially empty stack S?
  • Best case n, 1 for each operation.
  • Worst case O(n2), O(n) for each operation.
  • Average case complicate and difficult to
    analyze.
  • Amortized analysis 2n, 2 for each operation.
    (There are at most n push operations and hence at
    most n items popped out of the stack.)

41
The Difficulty of a Problem
  • Upper bound O(f(n)) means that for sufficiently
    large inputs, running time T(n) is bounded by a
    multiple of f(n)
  • Existing algorithms (upper bounds).
  • Lower bound W(f(n)) means that for sufficiently
    large n, there is at least one input of size n
    such that running time is at least a fraction of
    f(n) for any algorithm that solves the problem.
  • The inherent difficulty ? lower bound of
    algorithms
  • The lower bound of a method to solve a problem is
    not necessary the lower bound of the problem.

42
(No Transcript)
43
Examples
  • Sorting n elements into ascending order. 
  • O(n2), O(n?log n), etc. -- Upper bounds.
  • O(n), O(n?log n), etc. -- Lower bounds.
  • Lower bound matches upper bound.
  • Multiplication of 2 matrices of size n by n.
  • Straightforward algorithm O(n3).
  • Strassen's algorithm O(n2.81).
  • Best known sequential algorithm O(n2.376) ?
  • Best known lower bound W(n2)
  • The best algorithm for this problem is still open.

44
Complexity Classes
  • DSPACE(S(n)) NSPACE(S(n)) The classes of
    problems that can be solved by deterministic
    nondeterministic Turing machines using S(n)
    space. 
  • DTIME(T(n)) NTIME(T(n)) The classes of
    problems that can be solved by deterministic
    nondeterministic Turing machines using T(n)
    time.
  • Tractable problems Problems in P.
  •  Intractable problems Problem not known to be in
    P.
  •  Efficient algorithms Algorithms in P.

45
Complexity Classes
  • Assume P ? NP

NEXP
co-NEXP
EXP
PSPACE
NPC
P
co-NP
NP
46
NP-Complete Problems
  • M. R. Garey, and D. S. Johnson
  • Computers and Intractability A Guide to the
    Theory of NP-Completeness
  • W. H. Freeman and Company, 1979

47
Complexity of Algorithms and Problems
  • Notations
  • Symbol Meaning
  • P a problem
  • I a problem instance
  • In the set of all problem instances of size
  • A an algorithm for P
  • AP the set of algorithms for problem P
  • Pr(I) probability of instance I
  • CA(I) cost of A with input I
  • RA the set of all possible versions of a
    randomized algorithm A

48
Formal Definitions

f(n)
49
Example Complexity of the Sorting Problem
  • Assume comparison is used to determine the
    order of keys.

50
Divide-and-Conquer Algorithms Revisited
  • The divide-and-conquer paradigm
  • Divide the problem into a number of subproblems.
  • Conquer the subproblems (solve them).
  • Combine the subproblem solutions to get the
    solution to the original problem.
  • Merge sort T(n) 2T(n/2) ? (n) ? (n lg n).
  • Divide the n-element sequence to be sorted into
    two n/2-element sequence.
  • Conquer sort the subproblems, recursively using
    merge sort.
  • Combine merge the resulting two sorted
    n/2-element sequences.

51
Analyzing Divide-and-Conquer Algorithms
  • Recurrence for a divide-and-conquer algorithms
  • a of subproblems
  • n/b size of the subproblems
  • D(n) time to divide the problem of size n into
    subproblems
  • C(n) time to combine the subproblem solutions to
    get the answer for the problem of size n
  • Merge sort
  • a 2 two subproblems
  • n/b n/2 each subproblem has size ? n/2
  • D(n) ?(1) compute midpoint of array
  • C(n) ?(n) merging by scanning sorted subarrays

52
Divide-and-Conquer Binary Search
  • Binary search on a sorted array
  • Divide Check middle element.
  • Conquer Search the subarray.
  • Combine Trivial.
  • Recurrence T(n) T(n/2) ?(1) ?(lg n).
  • a 1 search one subarray
  • n/b n/2 each subproblem has size ? n/2
  • D(n) ?(1) compute midpoint of array
  • C(n) ?(1) trivial

53
Solving Recurrences
  • Three general methods for solving recurrences
  • Iteration Convert the recurrence into a
    summation by expanding some terms and then bound
    the summation
  • Substitution Guess a solution and verify it by
    induction.
  • Master Theorem if the recurrence has the form
  • T(n) aT(n/b) f(n),
  • then most likely there is a formula that
    can be applied.
  • Two simplifications that won't affect asymptotic
    analysis
  • Ignore floors and ceilings.
  • Assume base cases are constant, i.e., T(n) ?(1)
    for small n.

54
Solving Recurrences Iteration
  • Example T(n) 4T(n/2) n.

55
Iteration by Using Recursion Trees
  • Root computation (D(n) C(n)) at top level of
    recursion.
  • Node at level i Subproblem at level i in the
    recursion.
  • Height of tree level in the recursion.
  • T(n) sum of all nodes in the tree.
  • T(1)1 ? T(n) 4T(n/2) n n 2n 4n
    2lgnn ?(n2).

56
Solving Recurrences Substitution (Guess Verify)
  • Guess form of solution.
  • Apply math. induction to find the constant and
    verify solution.
  • Use to find an upper or a lower bound.
  • Example Guess T(n) 4T(n/2) n O(n3) (T(1)
    1)
  • Show T(n) ? cn3 for some c gt 0 (we must find c).
  • 1. Basis T(2) 4T(1) 2 6 ? 23c (pick c
    1)
  • 2. Assume T(k) ? ck3 for k lt n, and prove T(n) ?
    cn3
  • T(n) 4 T(n/2) n
  • ? 4 (c (n/2)3) n
  • cn3/2 n
  • cn3 - (cn3/2-n)
  • ? cn3,
  • where c ? 2 and n ? 1. (Pick c ?
    2 for Steps 1 2!)
  • Useful tricks subtract a lower order term,
    change variables (e.g., T(n)

57
Pitfall in Substitution
  • Example Guess T(n) 2T(n/2) n O(n) (wrong
    guess!)
  • Show T(n) ? cn for some c gt 0 (we must find c).
  • Basis T(2) 2T(1) 2 4 ? 2 c (pick c 2)
  • Assume T(k) ? ck for k lt n, and prove T(n) ? cn
  • T(n) 2 T(n/2) n
  • ? 2 (cn/2) n
  • cn n
  • O(n)
    / Wrong!! /
  • What's wrong?
  • How to fix? Subtracting a lower-order term may
    help!

58
Fixing Wrong Substitution
  • Guess T(n) 4T(n/2) n O(n2) (right guess!)
  • Assume T(k) ? ck2 for k lt n, and prove T(n) ?
    cn2.
  • T(n) 4T (n/2) n
  • ? 4c (n/2)2 n
  • cn2 n
  • O(n2)
    / Wrong!! /
  • Fix by subtracting a lower-order term.
  • Assume T(k) ? c1k2 - c2 k for k lt n, and prove
    T(n) ? c1 n2 - c2 n.
  • T(n) 4T(n/2) n
  • ? 4 (c1(n/2)2 -
    c2(n/2)) n
  • c1n2 - 2c2n n
  • ? c1n2 - c2n
    (if c2 ? 1)
  • Pick c1 big enough to handle initial conditions.

59
Solving Recurrence Relations
  • In general, we would prefer to have an explicit
    formula to compute the value of an rather than
    conducting n iterations.
  • For one class of recurrence relations, we can
    obtain such formulas in a systematic way.
  • Those are the recurrence relations that express
    the terms of a sequence as linear combinations of
    previous terms.

60
Solving Recurrence Relations
  • Definition A linear homogeneous recurrence
    relation of degree k with constant coefficients
    is a recurrence relation of the form
  • an c1an-1 c2an-2 ckan-k,
  • Where c1, c2, , ck are real numbers, and ck ? 0.
  • A sequence satisfying such a recurrence relation
    is uniquely determined by the recurrence relation
    and the k initial conditions
  • a0 C0, a1 C1, a2 C2, , ak-1 Ck-1.

61
Solving Recurrence Relations
  • Examples
  • The recurrence relation Pn (1.05)Pn-1
  • is a linear homogeneous recurrence relation of
    degree one.
  • The recurrence relation fn fn-1 fn-2
  • is a linear homogeneous recurrence relation of
    degree two.
  • The recurrence relation an an-5
  • is a linear homogeneous recurrence relation of
    degree five.

62
Solving Recurrence Relations
  • Basically, when solving such recurrence
    relations, we try to find solutions of the form
    an rn, where r is a constant.
  • an rn is a solution of the recurrence
    relationan c1an-1 c2an-2 ckan-k if and
    only if
  • rn c1rn-1 c2rn-2 ckrn-k.
  • Divide this equation by rn-k and subtract the
    right-hand side from the left
  • rk - c1rk-1 - c2rk-2 - - ck-1r - ck 0
  • This is called the characteristic equation of the
    recurrence relation.

63
Solving Recurrence Relations
  • The solutions of this equation are called the
    characteristic roots of the recurrence relation.
  • Let us consider linear homogeneous recurrence
    relations of degree two.
  • Theorem Let c1 and c2 be real numbers. Suppose
    that r2 c1r c2 0 has two distinct roots r1
    and r2.
  • Then the sequence an is a solution of the
    recurrence relation an c1an-1 c2an-2 if and
    only if an ?1r1n ?2r2n for n 0, 1, 2, ,
    where ?1 and ?2 are constants.

64
Solving Recurrence Relations
  • Example What is the solution of the recurrence
    relation an an-1 2an-2 with a0 2 and a1 7
    ?
  • Solution The characteristic equation of the
    recurrence relation is r2 r 2 0.
  • Its roots are r 2 and r -1.
  • Hence, the sequence an is a solution to the
    recurrence relation if and only if
  • an ?12n ?2(-1)n for some constants ?1 and
    ?2.

65
Solving Recurrence Relations
  • Given the equation an ?12n ?2(-1)n and the
    initial conditions a0 2 and a1 7, it follows
    that
  • a0 2 ?1 ?2
  • a1 7 ?1?2 ?2 ?(-1)
  • Solving these two equations gives us?1 3 and
    ?2 -1.
  • Therefore, the solution to the recurrence
    relation and initial conditions is the sequence
    an with
  • an 3?2n (-1)n.

66
Solving Recurrence Relations
  • an rn is a solution of the linear homogeneous
    recurrence relationan c1an-1 c2an-2
    ckan-k
  • if and only if
  • rn c1rn-1 c2rn-2 ckrn-k.
  • Divide this equation by rn-k and subtract the
    right-hand side from the left
  • rk - c1rk-1 - c2rk-2 - - ck-1r - ck 0
  • This is called the characteristic equation of the
    recurrence relation.

67
Solving Recurrence Relations
  • The solutions of this equation are called the
    characteristic roots of the recurrence relation.
  • Let us consider linear homogeneous recurrence
    relations of degree two.
  • Theorem Let c1 and c2 be real numbers. Suppose
    that r2 c1r c2 0 has two distinct roots r1
    and r2.
  • Then the sequence an is a solution of the
    recurrence relation an c1an-1 c2an-2 if and
    only if an ?1r1n ?2r2n for n 0, 1, 2, ,
    where ?1 and ?2 are constants.

68
Solving Recurrence Relations
  • Example Give an explicit formula for the
    Fibonacci numbers.
  • Solution The Fibonacci numbers satisfy the
    recurrence relation fn fn-1 fn-2 with initial
    conditions f0 0 and f1 1.
  • The characteristic equation is r2 r 1 0.
  • Its roots are

69
Solving Recurrence Relations
  • Therefore, the Fibonacci numbers are given by
  • for some constants ?1 and ?2.
  • We can determine values for these constants so
    that the sequence meets the conditions f0 0 and
    f1 1

70
Solving Recurrence Relations
  • The unique solution to this system of two
    equations and two variables is
  • So finally we obtained an explicit formula for
    the Fibonacci numbers

71
Solving Recurrence Relations
  • But what happens if the characteristic equation
    has only one root?
  • How can we then match our equation with the
    initial conditions a0 and a1 ?
  • Theorem Let c1 and c2 be real numbers with c2??
    0. Suppose that r2 c1r c2 0 has only one
    root r0. A sequence an is a solution of the
    recurrence relation an c1an-1 c2an-2 if and
    only if an ?1r0n ?2nr0n, for n 0, 1, 2, ,
    where ?1 and ?2 are constants.

72
Solving Recurrence Relations
  • Example What is the solution of the recurrence
    relation an 6an-1 9an-2 with a0 1 and a1
    6?
  • Solution The only root of r2 6r 9 0 is r0
    3.Hence, the solution to the recurrence
    relation is
  • an ?13n ?2n3n for some constants ?1 and ?2.
  • To match the initial condition, we need
  • a0 1 ?1a1 6 ?1?3 ?2?3
  • Solving these equations yields ?1 1 and ?2 1.
  • Consequently, the overall solution is given by
  • an 3n n3n.

73
Multiple Roots
  • If a is the root of the characteristic equation
    with multiplicity m, there is a general solution

74
Getting a Particular Solution
  • Mainly try and error. However, some very good
    suggestions do exist
  • Sometimes your initial guess "degenerates" and
    gives contradictory conditions. Then try a
    solution of higher degree.

75
Examples

76
A Total Example
  • Once the general and particular solutions have
    been found, they are added together to give the
    total solution.
  • The initial conditions are then used to determine
    the constants of the general solution.

77
A Total Example

78
Pseudo Nonlinear Recurrence
  • Range Transformations on the values of the
    sequences
  • Domain Transformations on the value of indices

79
Example

80
A Practical Example
  •  Mergesort
  • Split list in half
  • Sort both halves
  • Merge the two halves
  • Analysis The recurrence relation is
  • T(0) 0

81
Example Continued
  • T(n) 2T(n/2) (n 1)

82
Example Continued

83
Example Continued
  • (3) Impose the initial conditions on A2n n ? 2n
    1
  • (n0) A 0 1 U(0) 0 ? A -1
  • So U(n) n ? 2n 2n 1 T(2n)
  • Replace n by log n gives
  •  
  • T(n) n?log n n 1 Q(n ? log n)

84
Master Theorem
  • Let a ? 1 and b gt 1 be constants, f(n) be a
    function, and T(n) be defined on nonnegative
    integers as
  • T(n) aT(n/b) f(n).
  • Then, T(n) can be bounded asymptotically as
    follows

85
Solving Recurrences by Using Master Method
About PowerShow.com