Introduction%20to%20Algorithms%20(2nd%20edition)

1
Introduction to Algorithms(2nd edition)

• by Cormen, Leiserson, Rivest Stein
• Chapter 3 Growth of Functions
• (slides enhanced by N. Adlai A. DePano)

2
Overview

• Order of growth of functions provides a simple
  characterization of efficiency
• Allows for comparison of relative performance
  between alternative algorithms
• Concerned with asymptotic efficiency of
  algorithms
• Best asymptotic efficiency usually is best choice
  except for smaller inputs
• Several standard methods to simplify asymptotic
  analysis of algorithms

3
Asymptotic Notation

• Applies to functions whose domains are the set of
  natural numbersN 0,1,2,
• If time resource T(n) is being analyzed, the
  functions range is usually the set of
  non-negative real numbers T(n) ? R
• If space resource S(n) is being analyzed, the
  functions range is usually also the set of
  natural numbers S(n) ? N

4
Asymptotic Notation

• Depending on the textbook, asymptotic categories
  may be expressed in terms of --
• set membership (our textbook) functions belong
  to a family of functions that exhibit some
  property or
• function property (other textbooks) functions
  exhibit the property
• Caveat we will formally use (a) and informally
  use (b)

5
The T-Notation
T(g(n)) f(n) ?c1, c2 gt 0, n0 gt 0 s.t. ?n
n0 c1 g(n) f(n)
c2 g(n)
6
The O-Notation
O(g(n)) f(n) ?c gt 0, n0 gt 0 s.t. ?n n0
f(n) c g(n)
7
The O-Notation
O(g(n)) f(n) ?c gt 0, n0 gt 0 s.t. ?n n0
f(n) c g(n)
8
The o-Notation
o(g(n)) f(n) ?c gt 0 ?n0 gt 0 s.t. ?n n0
f(n) c g(n)
9
The ?-Notation
?(g(n)) f(n) ?c gt 0 ?n0 gt 0 s.t. ?n n0
f(n) c g(n)
10
Comparison of Functions
Transitivity

• f(n) O(g(n)) and g(n) O(h(n)) ? f(n)
  O(h(n))
• f(n) O(g(n)) and g(n) O(h(n)) ? f(n)
  O(h(n))
• f(n) T(g(n)) and g(n) T(h(n)) ? f(n)
  T(h(n))
• f(n) O(f(n)) f(n) O(f(n)) f(n) T(f(n))

Reflexivity
11
Comparison of Functions

• f(n) T(g(n)) ?? g(n) T(f(n))
• f(n) O(g(n)) ?? g(n) O(f(n))
• f(n) o(g(n)) ?? g(n) ?(f(n))
• f(n) O(g(n)) and f(n) O(g(n)) ? f(n)
  T(g(n))

Symmetry
Transpose Symmetry
Theorem 3.1
12
Asymptotic Analysis and Limits
13
Comparison of Functions

• f1(n) O(g1(n)) and f2(n) O(g2(n)) ?
  f1(n) f2(n) O(g1(n) g2(n))
• f(n) O(g(n)) ? f(n) g(n) O(g(n))

14
Standard Notation and Common Functions

• Monotonicity
• A function f(n) is monotonically increasing if m
  ? n implies f(m) ? f(n) .
• A function f(n) is monotonically decreasing if m
  ? n implies f(m) ? f(n) .
• A function f(n) is strictly increasing if m lt n
  implies f(m) lt f(n) .
• A function f(n) is strictly decreasing if m lt n
  implies f(m) gt f(n) .

15
Standard Notation and Common Functions

• Floors and ceilings
• For any real number x, the greatest integer less
  than or equal to x is denoted by ?x?.
• For any real number x, the least integer greater
  than or equal to x is denoted by ?x?.
• For all real numbers x, x?1 lt ?x? ? x ? ?x? lt
  x1.
• Both functions are monotonically increasing.

16
Standard Notation and Common Functions

• Exponentials
• For all n and a?1, the function an is the
  exponential function with base a and is
  monotonically increasing.
• Logarithms
• Textbook adopts the following convention lg n
  log2n (binary logarithm), ln n logen
  (natural logarithm), lgk n (lg n)k
  (exponentiation), lg lg n lg(lg n)
  (composition), lg n k (lg n)k (precedence
  of lg).

ai
17
Standard Notation and Common Functions

• Important relationships
• For all real constants a and b such that agt1,
  nb o(an) that is, any exponential function
  with a base strictly greater than unity grows
  faster than any polynomial function.
• For all real constants a and b such that agt0,
  lgbn o(na) that is, any positive polynomial
  function grows faster than any polylogarithmic
  function.

18
Standard Notation and Common Functions

• Factorials
• For all n the function n! or n factorial is
  given by n! n ? (n?1) ? (n ? 2) ? (n ? 3) ?
  ? 2 ? 1
• It can be established that n! o(nn) n!
  ?(2n) lg(n!) ?(nlgn)

19
Standard Notation and Common Functions

• Functional iteration
• The notation f (i)(n) represents the function
  f(n) iteratively applied i times to an initial
  value of n, or, recursively f (i)(n) n if
  n0 f (i)(n) f(f (i?1)(n)) if ngt0
• Example If f(n) 2n
• then f (2)(n) f(2n) 2(2n) 22n
• then f (3)(n) f(f (2)(n)) 2(22n)
  23n
• then f (i)(n) 2in

20
Standard Notation and Common Functions

• Iterated logarithmic function
• The notation lg n which reads log star of n is
  defined as lg n min i?0 lg(i) n ? 1
• Examplelg 2 1
• lg 4 2
• lg 16 3
• lg 65536 4
• lg 265536 5

21
Asymptotic Running Time of Algorithms

• We consider algorithm A better than algorithm B
  if TA(n) o(TB(n))
• Why is it acceptable to ignore the behavior of
  algorithms for small inputs?
• Why is it acceptable to ignore the constants?
• What do we gain by using asymptotic notation?

22
Things to Remember

• Asymptotic analysis studies how the values of
  functions compare as their arguments grow without
  bounds.
• Ignores constants and the behavior of the
  function for small arguments.
• Acceptable because all algorithms are fast for
  small inputs and growth of running time is more
  important than constant factors.

23
Things to Remember

• Ignoring the usually unimportant details, we
  obtain a representation that succinctly describes
  the growth of a function as its argument grows
  and thus allows us to make comparisons between
  algorithms in terms of their efficiency.