i206: Lecture 7: Analysis of Algorithms, continued

1
i206 Lecture 7Analysis of Algorithms, continued

• Marti Hearst
• Spring 2012

2
Algorithms

• A clearly defined procedure for solving a
  problem.
• A clearly described procedure is made up of
  simple clearly defined steps.
• The procedure can be complex, but the steps must
  be simple and unambiguous.

3
Problem Solving Strategies

• Try to work the problem backwards
• Reverse-engineer
• Once you know it can be done, it is much easier
  to do
• What are some examples?
• Stepwise Refinement
• Break the problem into several sub-problems
• Solve each sub-problem separately
• Produces a modular structure
• Look for a related problem that has been solved
  before
• Software design patterns

4
Example of Stepwise RefinementSpiderman

• Peter Parkers goal Make Mary Jane fall in love
  with him
• To accomplish this goal
• Get a cool car
• To accomplish this goal
• Get 3000
• To accomplish this goal
• Appear in a wrestling match
• Each goal requires completing just one subgoal

5
Example of Stepwise RefinementStar Wars Episode
IV

• Luke Skywalkers goal Make Princess Leia fall in
  love with him (they werent siblings yet)
• To accomplish this goal
• Rescue her from the death star
• To accomplish this goal
• Land on Death Star
• Remove her from her Prison Cell
• Disable the Tractor Beam
• Get her onto the Millennium Falcon
• To accomplish subgoal (2)
• Obtain Storm Trooper uniforms
• Have Wookie pose as arrested wild animal
• Find Location of Cell
• Have R2D2 communicate coordinates
• To accomplish subgoal (3)
• Have last living Jedi walk across catwalks
• To accomplish subgoal (4)
• Run down hall
• Survive in garbage shoot
• Fight garbage monster

6
Divide and Conquer Algorithms

• Break the problem into smaller pieces
• Recursively solve the sub-problems
• Combine the results into a full solution
• If structured properly, this can lead to a much
  faster solution.

7
Divide and Conquer Algorithms

• Problem Cut up the licorice into 64 pieces!.

8
Divide and Conquer Algorithms

• Problem Cut up the licorice into 64 pieces!
• How do we do this in
• O(n)
• O(log n)
• O(sqrt n)

9
Santas Socks

• Problem
• The elves have packed all 1024 boxes with skates
• But Santas dirty smelly socks fell into one of
  them.
• We have to find the box with the dirty socks!
  But its almost midnight!!!!

10
Santas Socks

• http//www.youtube.com/watch?vwVPCT1VjySA

11
Analysis of Algorithms

• Characterizing the running times of algorithms
  and data structure operations
• Secondarily, characterizing the space usage of
  algorithms and data structures

12
Algorithm Complexity

• Worst Case Complexity
• the function defined by the maximum number of
  steps taken on any instance of size n
• Best Case Complexity
• the function defined by the minimum number of
  steps taken on any instance of size n
• Average Case Complexity
• the function defined by the average number of
  steps taken on any instance of size n

Adapted from http//www.cs.sunysb.edu/algorith/le
ctures-good/node1.html
13
Best, Worst, and Average Case Complexity
Number of steps
Worst Case Complexity
Average Case Complexity
Best Case Complexity
N (input size)
Adapted from http//www.cs.sunysb.edu/algorith/le
ctures-good/node1.html
14
Monotonicity

• monotone (m n -t n )
• A succession of sounds or words uttered in a
  single tone of voice
• monotonic (m n -t n k) Mathematics.
  Designating sequences, the successive members of
  which either consistently increase or decrease
  but do not oscillate in relative value.
• A function is monotonically increasing if
  whenever agtb then f(a)gtf(b).
• A function is strictly increasing if whenever agtb
  then f(a)gtf(b).
• Similar for decreasing
• Nonmonotonic is the opposite

15
Monotonic and Nonmonotonic

• https//en.wikipedia.org/wiki/Monotonic_function

16
The Seven Common Functions

• Constant f(n) c
• E.g., adding two numbers, assigning a value to
  some variable, comparing two numbers, and other
  basic operations
• Useful free web tool Wolframs Alpha

17
The Seven Common Functions

• Linear f(n) n
• Do a single basic operation for each of n
  elements, e.g., reading n elements into computer
  memory, comparing a number x to each element of
  an array of size n

18
The Seven Common Functions

• Quadratic f(n) n2
• E.g., nested loops with inner and outer loops

Brookshear Figure 5.19
19
The Seven Common Functions

• Cubic f(n) n3

20
The Seven Common Functions

• More generally, all of the above are polynomial
  functions
• f(n) a0 a1n a2n2 a3n3 annd
• where the ais are constants, called the
  coefficients of the polynomial.

Brookshear Figure 5.19
21
The Seven Common Functions

• Logarithmic
• f(n) logbn
• Intuition number of times we can divide n by b
  until we get a number less than or equal to 1
• The base is omitted for the case of b2, which is
  the most common in computing
• log n log2n

Brookshear Figure 5.20
22
The Seven Common Functions

• n-log-n function f(n) n log n
• Product of linear and logarithmic

23
Comparing Growth Rates

• 1 lt log n lt n1/2 lt n lt n log n lt n2 lt n3 lt bn

http//www.cs.pomona.edu/marshall/courses/2002/sp
ring/cs50/BigO/
24
Polynomials

• Only the dominant terms of a polynomial matter in
  the long run.  Lower-order terms fade to
  insignificance as the problem size increases.

http//www.cs.pomona.edu/marshall/courses/2002/sp
ring/cs50/BigO/
25
Definition of Big-Oh A running time is O(g(n))
if there exist constants n0 gt 0 and c gt 0 such
that for all problem sizes n gt n0, the running
time for a problem of size n is at most
c(g(n)). In other words, c(g(n)) is an upper
bound on the running time for sufficiently large
n.
c g(n)
Example the function f(n)8n2 is O(n) The
running time of algorithm arrayMax is O(n)
http//www.cs.dartmouth.edu/farid/teaching/cs15/c
s5/lectures/0519/0519.html
26
More formally

• Let f(n) and g(n) be functions mapping
  nonnegative integers to real numbers.
• f(n) is ?(g(n)) if there exist positive constants
  n0 and c such that for all ngtn0, f(n) lt cg(n)
  
• Other ways to say this
• f(n) is order g(n)
• f(n) is big-Oh of g(n)
• f(n) is Oh of g(n)
• f(n) ? O(g(n)) (set notation)

27
Comparing Running Times
Adapted from Goodrich Tamassia
28
Analysis Example Phonebook

• Given
• A physical phone book
• Organized in alphabetical order
• A name you want to look up
• An algorithm in which you search through the book
  sequentially, from first page to last
• What is the order of
• The best case running time?
• The worst case running time?
• The average case running time?
• What is
• A better algorithm?
• The worst case running time for this algorithm?

29
Analysis Example (Phonebook)

• This better algorithm is called Binary Search
• What is its running time?
• First you look in the middle of n elements
• Then you look in the middle of n/2 ½n elements
• Then you look in the middle of ½ ½n elements
• Continue until there is only 1 element left
• Say you did this m times ½ ½ ½ n
• Then the number of repetitions is the smallest
  integer m such that

30
Analyzing Binary Search

• In the worst case, the number of repetitions is
  the smallest integer m such that
• We can rewrite this as follows

Multiply both sides by
Take the log of both sides
Since m is the worst case time, the algorithm is
O(logn)
31
Hint for Homework Problem 1

• Hint 1
• Write a program that computes the number of gifts
  for each value of N just to get a feeling for how
  it works.
• Hint 2
• Last time we showed how triangles of dots prove
•     1 2 3 ... n      n(n 1)/2
• Similarly, to add up the number of gifts, a
  closed form version of the total numbers is
• (n/6)(n1)(n2)

32
Summary Analysis of Algorithms

• A method for determining, in an abstract way, the
  asymptotic running time of an algorithm
• Here asymptotic means as n gets very large
• Useful for comparing algorithms
• Useful also for determing tractability
• Meaning, a way to determine if the problem is
  intractable (impossible) or not
• Exponential time algorithms are usually
  intractable.
• Well revisit these ideas throughout the rest of
  the course.