Complexity of Algorithms

1
Complexity of Algorithms
2
Learning Objectives

• Understand what is complexity of algorithms.
• Practice how to calculate complexity of
  algorithms.

3
Complexity of Algorithms

• Time Complexity Determine the approximate number
  of operations required to solve a problem of size
  n (number of comparisons, or number of basic
  operations, such as addition, multiplication, )
• Space Complexity Determine the approximate
  memory required to solve a problem of size n.
• Time Complexity
• - Use the Big-O notation
• - Ignore house keeping
• - Count the expensive operations only

4
Complexity of Algorithms

• Basic operations
• searching algorithms - key comparisons
• sorting algorithms - list component comparisons
• numerical algorithms - floating point ops.
  (flops) -multiplications/divisions and/or
  additions/subtractions
• Worst Case maximum number of operations
• Average Case mean number of operations assuming
  an input probability distribution
• Best Case minimum number of operations

5
Complexity of Algorithms

• Example maximum elementuse number of
  comparisons
• 1 comparison end of list 1 comparison with max
• so 2 comparisons at every iteration
• at the end of list one last comparison
• number of iterations from 2 to n gives (n-1)
  iterations
• total number of comparisons 2(n-1) 1
• O(n) (Worst case average case)
• O(1) (Best case)

6
Complexity of Algorithms

• Examples
• Multiply an n x n matrix A by a scalar c to
  produce the matrix B
• procedure (n, c, A, B integers)
• for i 1 to n do
• for j 1 to n do
• B(i, j) cA(i, j)
• end do
• end do
• Analysis (worst case, average case, best case)
• Count the number of floating point
  multiplications.
• n 2 elements requires n 2 multiplications. time
  complexity is
• O(n 2 ) or quadratic complexity.

7
Complexity of Algorithms

• Bubble sort L is a list of elements to be
  sorted.
• - We assume nothing about the initial order
• - The list is in ascending order upon
  completion.
• Analysis (worst case)
• Count the number of list comparisons required.
• Method If the jth element of L is larger than
  the (j 1)st, swap them.
• Note this is not an efficient implementation of
  the algorithm

8
Complexity of Algorithms
9
Complexity of Algorithms

• Bubble the largest element to the 'top' by
  starting at the bottom - swap elements until the
  largest in the top position. Bubble the second
  largest to the position below the top.Continue
  until the list is sorted.
• n-1 comparison on the first pass
• n-2 comparisons on the second pass
• 1 comparison on the last pass
• Total (n - 1) (n - 2) . . . . 1 n
  (n-1) / 2 O(n 2 ) (quadratic complexity)
• (what is the leading coefficient?)
• The total algorithm (outer and inner loops) is
  in O(n2).

10
Complexity of Algorithms

• An algorithm to determine if a function f from A
  to B is an injection
• Input a table with two columns
• - Left column contains the elements of A.
• - Right column contains the images of the
  elements in the left column.
• Analysis (worst case)
• Count comparisons of elements of B.
• Recall that two elements of column 1 cannot have
  the same images in column 2.

11
Complexity of Algorithms

• One solution
• Sort the right column
• Worst case complexity (using Bubble sort)
• O(n 2 )
• Compare adjacent elements to see if they agree
• Worst case complexity
• O(n)
• Total
• O(n 2 ) O(n) O(n 2 )

12
Algorithmic Analysis

• Better Characterizations using Less Information
• Question Why are dictionaries ordered?
• Answer So we can use a more efficient algorithm
  to look up words.
• Why is it more efficient? To search an unordered
  list of n words takes, worst case, n comparisons.
  To search an ordered list requires, worst case,
  log n comparisons (using binary search, as we do
  with dictionaries).
• Note that we ignore features such as how long it
  actually takes to do a comparison, as the exact
  figure is relatively unimportant.

13
Algorithmic Analysis

• Asymptotic Execution Time
• Suppose f(n) is a formula that describes the
  exact execution time of some algorithm for inputs
  of size n.
• We then determine whether f(n) is Big-O of a
  well-known function.
• Searching a dictionary is O(log n), for example,
  while searching an unordered list is O(n).
• This chapter is devoted to showing how to
  characterize the asymptotic execution time
  behavior of algorithms.

14
Summing Execution Time

• When summing execution times, the dominant
  function is the only important one.
• procedure makeIdentityMatrix (mnn matrix,
  n integer)
• for i0 to n-1 do
• for j 0 to n-1 do
• mi,j 0.0
• end do
• end do
• for i 0 to n-1 do mi,i 1.0 end
  do
• m is the identity matrix

O(n2)
15
Ranking of Execution Time
16
Summing Execution Time

• Comparison of execution times

17
Summing Execution Time

• When Adding Figures, Larger Eventually Dominates