Algorithms CSCI 235, Fall 2005 Lecture 2 Introduction to Asymptotic Analysis Read Ch' 2 and 3 of Tex

1
Algorithms CSCI 235, Fall 2005Lecture
2Introduction to Asymptotic AnalysisRead Ch. 2
and 3 of Text
2
Recall Insertion Sort
Idea Sort items one at a time into previously
sorted group. Invariant After the ith step of
the algorithm, A1..i is sorted.
3
Pseudocode for Insertion Sort

• Insertion-Sort(A)
• for j lt- 2 to lengthA
• do key lt- Aj
• i lt- j - 1
• while i gt 0 and Ai gt key
• do Ai1 lt- Ai
• i lt- i - 1
• Ai1 lt- key

Cost c1 c2 c3 c4 c5 c6 c7
times n n-1 n-1 ? ? ? n-1
4
Analyzing the while loop
Let tj number of times the while loop test is
executed for a given value of j. tj depends on
the initial order of the array. Total number of
times the while loop test is executed
The body of the while loop is executed one less
time than the while loop test for each value of j
5
Pseudocode for Insertion Sort

• Insertion-Sort(A)
• for j lt- 2 to lengthA
• do key lt- Aj
• i lt- j - 1
• while i gt 0 and Ai gt key
• do Ai1 lt- Ai
• i lt- i - 1
• Ai1 lt- key

Cost c1 c2 c3 c4 c5 c6 c7
times n n-1 n-1 n-1
6
What is tj? Best Case

• Best Case All items in sorted order
• Nothing needs to be shifted
• While loop test executed once for each j
• Body executed 0 times for each j

test execution
0
body execution
7
Best Case--Adding it up

• Insertion-Sort(A)
• for j lt- 2 to lengthA
• do key lt- Aj
• i lt- j - 1
• while i gt 0 and Ai gt key
• do Ai1 lt- Ai
• i lt- i - 1
• Ai1 lt- key

Cost c1 c2 c3 c4 c5 c6 c7
times n n-1 n-1 n-1 0 0 n-1
T(n) c1n c2(n-1) c3(n-1) c4(n-1)
c7(n-1) (c1 c2 c3 c4 c7) n - (c2 c3
c4 c7) an - b
Best Case is Linear!
8
What is tj? Worst Case

• Worst Case All items in reverse sorted order
• Must look through all previously sorted items to
  place each item.
• While loop test executed j times for each j
• Body executed j-1 times for each j

test execution
body execution
9
Worst Case--Adding it up

• Insertion-Sort(A)
• for j lt- 2 to lengthA
• do key lt- Aj
• i lt- j - 1
• while i gt 0 and Ai gt key
• do Ai1 lt- Ai
• i lt- i - 1
• Ai1 lt- key

Cost c1 c2 c3 c4 c5 c6 c7
times n n-1 n-1 n(n1)/2-1 n(n-1)/2 n(n-1)/2 n-1
T(n) (c4/2 c5/2 c6/2)n2 (c1 c2 c3
c4/2 - c5/2 - c6/2 c7) n - (c2 c3 c4 c7)
an2 bn c
Worst Case is quadratic!
10
Recall Merge Sort
Divide in 1/2 repeatedly
merge in sorted order
T(n) c1n lg(n) c1n
How does this compare to Insertion sort?
Depends on a, b, c, c1 and n
For large enough n, Merge sort is always faster.
11
Asymptotic analysis

• Measuring Growth Rates
• When comparing the running time of two
  algorithms, a good measure is how fast the
  running time increases as the input size
  increases.
• When comparing growth rates, we can ignore all
  but the highest order term.
• When comparing growth rates, we can ignore
  constant coefficients.
• We would like a coarse way to compare functions
  that ignores constants and focuses on their
  relative growth as the input size gets large.

12
Examples of Growth Rate

• n1 n1000 n1,000,000
• p(n) 100n 1000
• q(n) 3n2 2n 1
• r(n) 0.1n2

1100 101000 100,001,000108
6 3,002,001 31012210611012
0.1 100,000 1011
Lower order terms become insignificant as n
increases. r and q are quadratic. They grow much
faster than p. Asymptotic notation gives us a way
to express this relationship p is o(r), i.e. p
grows slower than r. p is also O(r), i.e. p
grows no faster than r. r is w(p), i.e. r grows
faster than p. r is also W(p), i.e. r grows no
slower than p. r is Q(q), i.e. r and q grow at
the same rate. Next time we will learn the
mathematical definitions of these.