Loading...

PPT – i206: Lecture 7: Analysis of Algorithms, continued PowerPoint presentation | free to download - id: 7bc888-NGEyM

The Adobe Flash plugin is needed to view this content

i206 Lecture 7Analysis of Algorithms, continued

- Marti Hearst
- Spring 2012

Algorithms

- A clearly defined procedure for solving a

problem. - A clearly described procedure is made up of

simple clearly defined steps. - The procedure can be complex, but the steps must

be simple and unambiguous.

Problem Solving Strategies

- Try to work the problem backwards
- Reverse-engineer
- Once you know it can be done, it is much easier

to do - What are some examples?
- Stepwise Refinement
- Break the problem into several sub-problems
- Solve each sub-problem separately
- Produces a modular structure
- Look for a related problem that has been solved

before - Software design patterns

Example of Stepwise RefinementSpiderman

- Peter Parkers goal Make Mary Jane fall in love

with him - To accomplish this goal
- Get a cool car
- To accomplish this goal
- Get 3000
- To accomplish this goal
- Appear in a wrestling match
- Each goal requires completing just one subgoal

Example of Stepwise RefinementStar Wars Episode

IV

- Luke Skywalkers goal Make Princess Leia fall in

love with him (they werent siblings yet) - To accomplish this goal
- Rescue her from the death star
- To accomplish this goal
- Land on Death Star
- Remove her from her Prison Cell
- Disable the Tractor Beam
- Get her onto the Millennium Falcon
- To accomplish subgoal (2)
- Obtain Storm Trooper uniforms
- Have Wookie pose as arrested wild animal
- Find Location of Cell
- Have R2D2 communicate coordinates
- To accomplish subgoal (3)
- Have last living Jedi walk across catwalks
- To accomplish subgoal (4)
- Run down hall
- Survive in garbage shoot
- Fight garbage monster

Divide and Conquer Algorithms

- Break the problem into smaller pieces
- Recursively solve the sub-problems
- Combine the results into a full solution
- If structured properly, this can lead to a much

faster solution.

Divide and Conquer Algorithms

- Problem Cut up the licorice into 64 pieces!.

Divide and Conquer Algorithms

- Problem Cut up the licorice into 64 pieces!
- How do we do this in
- O(n)
- O(log n)
- O(sqrt n)

Santas Socks

- Problem
- The elves have packed all 1024 boxes with skates
- But Santas dirty smelly socks fell into one of

them. - We have to find the box with the dirty socks!

But its almost midnight!!!!

Santas Socks

- http//www.youtube.com/watch?vwVPCT1VjySA

Analysis of Algorithms

- Characterizing the running times of algorithms

and data structure operations - Secondarily, characterizing the space usage of

algorithms and data structures

Algorithm Complexity

- Worst Case Complexity
- the function defined by the maximum number of

steps taken on any instance of size n - Best Case Complexity
- the function defined by the minimum number of

steps taken on any instance of size n - Average Case Complexity
- the function defined by the average number of

steps taken on any instance of size n

Adapted from http//www.cs.sunysb.edu/algorith/le

ctures-good/node1.html

Best, Worst, and Average Case Complexity

Number of steps

Worst Case Complexity

Average Case Complexity

Best Case Complexity

N (input size)

Adapted from http//www.cs.sunysb.edu/algorith/le

ctures-good/node1.html

Monotonicity

- monotone (m n -t n )
- A succession of sounds or words uttered in a

single tone of voice - monotonic (m n -t n k) Mathematics.

Designating sequences, the successive members of

which either consistently increase or decrease

but do not oscillate in relative value. - A function is monotonically increasing if

whenever agtb then f(a)gtf(b). - A function is strictly increasing if whenever agtb

then f(a)gtf(b). - Similar for decreasing
- Nonmonotonic is the opposite

Monotonic and Nonmonotonic

- https//en.wikipedia.org/wiki/Monotonic_function

The Seven Common Functions

- Constant f(n) c
- E.g., adding two numbers, assigning a value to

some variable, comparing two numbers, and other

basic operations - Useful free web tool Wolframs Alpha

The Seven Common Functions

- Linear f(n) n
- Do a single basic operation for each of n

elements, e.g., reading n elements into computer

memory, comparing a number x to each element of

an array of size n

The Seven Common Functions

- Quadratic f(n) n2
- E.g., nested loops with inner and outer loops

Brookshear Figure 5.19

The Seven Common Functions

- Cubic f(n) n3

The Seven Common Functions

- More generally, all of the above are polynomial

functions - f(n) a0 a1n a2n2 a3n3 annd
- where the ais are constants, called the

coefficients of the polynomial.

Brookshear Figure 5.19

The Seven Common Functions

- Logarithmic
- f(n) logbn
- Intuition number of times we can divide n by b

until we get a number less than or equal to 1 - The base is omitted for the case of b2, which is

the most common in computing - log n log2n

Brookshear Figure 5.20

The Seven Common Functions

- n-log-n function f(n) n log n
- Product of linear and logarithmic

Comparing Growth Rates

- 1 lt log n lt n1/2 lt n lt n log n lt n2 lt n3 lt bn

http//www.cs.pomona.edu/marshall/courses/2002/sp

ring/cs50/BigO/

Polynomials

- Only the dominant terms of a polynomial matter in

the long run. Lower-order terms fade to

insignificance as the problem size increases.

http//www.cs.pomona.edu/marshall/courses/2002/sp

ring/cs50/BigO/

Definition of Big-Oh A running time is O(g(n))

if there exist constants n0 gt 0 and c gt 0 such

that for all problem sizes n gt n0, the running

time for a problem of size n is at most

c(g(n)). In other words, c(g(n)) is an upper

bound on the running time for sufficiently large

n.

c g(n)

Example the function f(n)8n2 is O(n) The

running time of algorithm arrayMax is O(n)

http//www.cs.dartmouth.edu/farid/teaching/cs15/c

s5/lectures/0519/0519.html

More formally

- Let f(n) and g(n) be functions mapping

nonnegative integers to real numbers. - f(n) is ?(g(n)) if there exist positive constants

n0 and c such that for all ngtn0, f(n) lt cg(n)

- Other ways to say this
- f(n) is order g(n)
- f(n) is big-Oh of g(n)
- f(n) is Oh of g(n)
- f(n) ? O(g(n)) (set notation)

Comparing Running Times

Adapted from Goodrich Tamassia

Analysis Example Phonebook

- Given
- A physical phone book
- Organized in alphabetical order
- A name you want to look up
- An algorithm in which you search through the book

sequentially, from first page to last - What is the order of
- The best case running time?
- The worst case running time?
- The average case running time?
- What is
- A better algorithm?
- The worst case running time for this algorithm?

Analysis Example (Phonebook)

- This better algorithm is called Binary Search
- What is its running time?
- First you look in the middle of n elements
- Then you look in the middle of n/2 ½n elements
- Then you look in the middle of ½ ½n elements
- Continue until there is only 1 element left
- Say you did this m times ½ ½ ½ n
- Then the number of repetitions is the smallest

integer m such that

Analyzing Binary Search

- In the worst case, the number of repetitions is

the smallest integer m such that - We can rewrite this as follows

Multiply both sides by

Take the log of both sides

Since m is the worst case time, the algorithm is

O(logn)

Hint for Homework Problem 1

- Hint 1
- Write a program that computes the number of gifts

for each value of N just to get a feeling for how

it works. - Hint 2
- Last time we showed how triangles of dots prove
- 1 2 3 ... n n(n 1)/2
- Similarly, to add up the number of gifts, a

closed form version of the total numbers is - (n/6)(n1)(n2)

Summary Analysis of Algorithms

- A method for determining, in an abstract way, the

asymptotic running time of an algorithm - Here asymptotic means as n gets very large
- Useful for comparing algorithms
- Useful also for determing tractability
- Meaning, a way to determine if the problem is

intractable (impossible) or not - Exponential time algorithms are usually

intractable. - Well revisit these ideas throughout the rest of

the course.