Analysis of Algorithm Efficiency - PowerPoint PPT Presentation

About This Presentation
Title:

Analysis of Algorithm Efficiency

Description:

Analysis of Algorithm Efficiency Dr. Yingwu Zhu p5-11, p16-29, p43-53, p93-96 – PowerPoint PPT presentation

Number of Views:345
Avg rating:3.0/5.0
Slides: 75
Provided by: zhu120
Category:

less

Transcript and Presenter's Notes

Title: Analysis of Algorithm Efficiency


1
Analysis of Algorithm Efficiency
  • Dr. Yingwu Zhu
  • p5-11, p16-29, p43-53, p93-96

2
What is Algorithm?
  • Any well-defined computational procedure that
  • takes input some value or a set of values
  • and
  • produces output some value or a set of values
  • Example sorting problems
  • Input a sequence of numbers a1, a2, , an
  • Output a permutation b1, b2, , bn of the
    input s.t. b1ltb2ltltbn

3
Algorithms Motivation
  • At the heart of programs lie algorithms
  • Algorithms as a key technology
  • Think about
  • mapping/navigation
  • Google search
  • word processing (spelling correction, layout)
  • content delivery and streaming video
  • games (graphics, rendering)
  • big data (querying, learning)

4
Algorithms Motivation
  • In a perfect world
  • for each problem we would have an algorithm
  • the algorithm would be the fastest possible
  • What would CS look like in this world?

5
Algorithms Motivation
  • Our world (fortunately) is not so perfect
  • for many problems we know embarrassingly little
    about what the fastest algorithm is
  • multiplying two integers or two matrices
  • factoring an integer into primes
  • determining shortest tour of given n cities
  • for many problems we suspect fast algorithms are
    impossible (NP-complete problems)
  • for some problems we have unexpected and clever
    algorithms (we will see many of these)

6
Analyzing Algorithms
  • Given computing resources and input data
  • how fast does an algorithm run?
  • Time efficiency amount of time required to
    accomplish the task
  • Our focus
  • How much memory is required?
  • Space efficiency amount of memory required
  • Deals with the extra space the algorithm requires

7
Time Efficiency
  • Time efficiency depends on
  • size of input
  • speed of machine
  • quality of source code
  • quality of compiler

So, we cannot express time efficiency
meaningfully in real time units such as seconds!
8
Empirical analysis of time efficiency
  • Select a specific (typical) sample of inputs
  • Use physical unit of time (e.g., milliseconds)
  • or
  • Count actual number of basic operations
    executions
  • Analyze the empirical data
  • Limitation results dependent on the particular
    computer and the sample of inputs

9
Theoretical analysis of time efficiency
  • Time efficiency is analyzed by determining the
    number of repetitions of the basic operation as a
    function of input size
  • Basic operation the operation that contributes
    most towards the running time of the algorithm
  • T(n) copC(n)

10
Input size and basic operation examples
Problem Input size measure Basic operation
Searching for key in a list of n items Number of lists items, i.e. n Key comparison
Multiplication of two matrices Matrix dimensions or total number of elements Multiplication of two numbers
Checking primality of a given integer n nsize number of digits (in binary representation) Division
Typical graph problem vertices and/or edges Visiting a vertex or traversing an edge
11
Time Efficiency
  • T(n) (approximated by) number of times the
    basic operation is executed.
  • Not only depends on the input size n, but also
    depends on the arrangement of the input items
  • Best case not informative
  • Average case difficult to determine
  • Worst case is used to measure an algorithms
    performance

12
Example Sequential Search
  • Best case T(n)
  • Worst case T(n)
  • Average case T(n)
  • Assume success search probability of p

13
Order of Growth
  • Established framework for analyzing T(n)
  • Order of growth as n?8
  • Highest-order term is what counts
  • Remember, we are doing asymptotic analysis
  • As the input size grows larger it is the high
    order term that dominates
  • disregard lower-order terms in running time
  • disregard coefficient on highest order term
  • Asymptotic notations ?, O, ?

14
Asymptotic Order of Growth
  • A way of comparing functions that ignores
    constant factors and small input sizes
  • O(g(n)) class of functions f(n) that grow no
    faster than g(n)
  • T(g(n)) class of functions f(n) that grow at
    same rate as g(n)
  • O(g(n)) class of functions f(n) that grow at
    least as fast as g(n)

15
Big-oh
16
Big-omega
17
Big-theta
18
Upper Bound Notation
  • In general a function
  • f(n) is O(g(n)) if there exist positive constants
    c and n0 such that f(n) ? c ? g(n) for all n ? n0
  • Order of growth of f(n) ? order of growth of g(n)
    (within constant multiple)
  • Formally
  • O(g(n)) f(n) ? positive constants c and n0
    such that f(n) ? c ? g(n) ? n ? n0

19
Big-Oh
  • Examples
  • 10n is O(n2)
  • 5n20 is O(n)

20
An Example Insertion Sort
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

21
An Example Insertion Sort
i ? j ? key ?Aj ? Aj1 ?
30
10
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

22
An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 10
30
10
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

23
An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 30
30
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

24
An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 30
30
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

25
An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 30
30
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

26
An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 30
30
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

27
An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 10
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

28
An Example Insertion Sort
i 3 j 0 key 10Aj ? Aj1 10
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

29
An Example Insertion Sort
i 3 j 0 key 40Aj ? Aj1 10
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

30
An Example Insertion Sort
i 3 j 0 key 40Aj ? Aj1 10
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

31
An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

32
An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

33
An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

34
An Example Insertion Sort
i 4 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

35
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

36
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

37
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 20
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

38
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 20
10
30
40
20
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

39
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 40
10
30
40
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

40
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 40
10
30
40
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

41
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 40
10
30
40
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

42
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

43
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

44
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 30
10
30
30
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

45
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 30
10
30
30
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

46
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 30
10
30
30
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

47
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 30
10
30
30
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

48
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 20
10
20
30
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

49
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 20
10
20
30
40
1
2
3
4
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

Done!
50
Insertion Sort
  • InsertionSort(A, n) for i 2 to n key
    Ai j i - 1 while (j gt 0) and (Aj gt key)
    Aj1 Aj j j - 1 Aj1
    key

How many times will this loop execute?
51
T(n) for Insertion Sort
  • Worst case?
  • Best case?

52
Insertion Sort Is O(n2)
  • Proof
  • Suppose runtime is an2 bn c
  • If any of a, b, and c are less than 0 replace
    the constant with its absolute value
  • an2 bn c ? (a b c)n2 (a b c)n (a
    b c)
  • ? 3(a b c)n2 for n ? 1
  • Let c 3(a b c) and let n0 1
  • Question
  • Is InsertionSort O(n3)?
  • Is InsertionSort O(n)?

53
Big O Fact
  • A polynomial of degree k is O(nk)
  • Proof
  • Suppose f(n) bknk bk-1nk-1 b1n b0
  • Let ai bi
  • f(n) ? aknk ak-1nk-1 a1n a0

54
Lower Bound Notation
  • In general a function
  • f(n) is ?(g(n)) if ? positive constants c and n0
    such that 0 ? c?g(n) ? f(n) ? n ? n0
  • Proof
  • Suppose run time is an b
  • Assume a and b are positive (what if b is
    negative?)
  • an ? an b
  • Example n3 ?(n2)

55
Asymptotic Tight Bound
  • A function f(n) is ?(g(n)) if ? positive
    constants c1, c2, and n0 such that c1 g(n) ?
    f(n) ? c2 g(n) ? n ? n0
  • Theorem
  • f(n) is ?(g(n)) iff f(n) is both O(g(n)) and
    ?(g(n))
  • Proof
  • Engineering
  • Drop low-order items ignore leading constants
  • Example 3n3 90n2 -5n 9999 ?(n3)

56
Using limits for comparing orders of growth
T(n) has a smaller order of growth than g(n)
T(n) has the same order of growth than g(n)
T(n) has a larger order of growth than g(n)
Example compare the orders of growth of
and n2
57
Practical Complexity
58
Practical Complexity
59
Practical Complexity
60
Practical Complexity
61
Properties of Big-O
  • f(n) ? O(f(n))
  • f(n) ? O(g(n)) iff g(n) ??(f(n))
  • If f (n) ? O(g (n)) and g(n) ? O(h(n)) , then
    f(n) ? O(h(n)) Note similarity with a b
  • If f1(n) ? O(g1(n)) and f2(n) ? O(g2(n)) , then
  • f1(n) f2(n) ? O(maxg1(n),
    g2(n))

62
Basic asymptotic efficiency classes
1 constant
log n logarithmic
n linear
n log n n-log-n
n2 quadratic
n3 cubic
2n exponential
n! factorial
63
Summary Analysis for Algorithms
  • Focus on worst case
  • well-suited to rigorous analysis, simple
  • Measure time/space complexity using asymptotic
    notation (big-oh notation)
  • disregard lower-order terms in running time
  • disregard coefficient on highest order term
  • Non-recursive algorithms
  • Recursive algorithms

64
T(n) for nonrecursive algorithms
  • General Plan for Analysis
  • Decide on parameter n indicating input size
  • Identify algorithms basic operation
  • Determine worst, average, and best cases for
    input of size n
  • Set up a sum for the number of times the basic
    operation is executed
  • Simplify the sum using standard formulas and
    rules
  • Disregard low-order terms coefficient on the
    highest-order term

65
Useful summation formulas and rules
  • ?l?i?u1 111 u - l 1
  • In particular, ?1?i?n1 n - 1 1 n ?
    ?(n)
  • ?1?i?n i 12n n(n1)/2 ? n2/2 ? ?(n2)
  • ?1?i?n i2 1222n2 n(n1)(2n1)/6 ? n3/3 ?
    ?(n3)
  • ?0?i?n ai 1 a an (an1 - 1)/(a - 1)
    for any a ? 1
  • In particular, ?0?i?n 2i 20 21
    2n 2n1 - 1 ? ?(2n )
  • ?(ai bi ) ?ai ?bi ?cai c?ai
    ?l?i?uai ?l?i?mai ?m1?i?uai

66
Example 1 Maximum element
67
Example 2 Element uniqueness problem
68
Example 3 Matrix multiplication
69
Plan for Analysis of Recursive Algorithms
  • Decide on a parameter indicating an inputs
    size.
  • Identify the algorithms basic operation.
  • Check whether the number of times the basic op.
    is executed may vary on different inputs of the
    same size. (If it may, the worst, average, and
    best cases must be investigated separately.)
  • Set up a recurrence relation with an appropriate
    initial condition expressing the number of times
    the basic op. is executed.
  • Solve the recurrence (or, at the very least,
    establish its solutions order of growth) by
    backward substitutions or another method.

70
Example 1 Recursive evaluation of n!
  • Definition n ! 1 ? 2 ? ?(n-1) ? n for n 1
    and 0! 1
  • Recursive definition of n! F(n) F(n-1) ? n
    for n 1 and

  • F(0) 1
  • Size
  • Basic operation
  • Recurrence relation

71
Example 2 The Tower of Hanoi Puzzle


Recurrence for number of moves
n disks of different sizes and 3 pegs the goal
is to move all the disks to the third peg using
the 2nd one as an auxiliary if necessary. Move
on disk at a time and donot place a larger disk
on top of a smaller one!
72
The Tower of Hanoi Puzzle
  • T(n) 2T(n-1) 1
  • T(1) 1

73
Example 3 Binary Search
74
Master Theorem
  • T(n) aT(n/b) f (n), where f(n) ? ?(nd),
    d ? 0
  • Master Theorem
    If a lt bd, T(n) ? ?(nd)
  • If a bd,
    T(n) ? ?(nd log n)
  • If a gt bd,
    T(n) ? ?(nlog ba )
Write a Comment
User Comments (0)
About PowerShow.com