Complexity

1
Complexity

• Present sorting methods. Binary search. Other
  measures.
• Homework presentations. Postings.

2
Big Oh mechanics

• Big Oh defines an upper bound. You get to pick a
  coefficient c (multiplier) and you get to pick an
  n beyond which the bound holds.
• Example f(n) n 1000 is bounded by n and also
  by n2.
• For O(n), try c 1001 and n0 1.
• (n1000) lt 1001 (n)
• For O(n2), try c1001 and n0 1
• (n1000) lt 1001(n2)

3
Big Oh

• But can we find a c and a n0 to make n2 be
  gounded by n?
• Answer no.
• Because no matter what the coefficient c, let n
  be bigger than it, and n2 gt n.

4
Another example

• n2n1 is O(n2)
• set c 4, n0 1, then(n2n1) lt 4 (n2) or
• set c 3, n0 2, then (n2n1) lt 3 (n2)
  starting with n 2
• NOTE don't worry about the c and the n0 being
  the best (tightest) bounds.

5
Bounds

• Distinction sometimes made between
• mean bound
• average bound
• achieved bound
• There also is omega notation.
• posting opportunity

6
Refresher on log

• Definition logbN is defined to be the value e
  such that be is equal to N.
• When using log for Big Oh, it doesn't matter what
  the base is because logs to different bases
  differ by a coefficient.
• log210 log10N is equal to log2N.
• Proof raise 2 to each
• Right side 2 log210 log10N is (2 log210 )
  log10N is
• 10 log10N is N is the same as left side.

7
Compare logN and N

• log N is bound by N, that is, always less than N.
• Order
• O(c) constant less than O(logN) less than O(N)
  etc.

8
Sorts

• Sorts that are O(n2) when n is the size of the
  data grow much faster than sorts that are O(N
  logN )
• Demonstrate?

9
O(N) linear algorithm

• Data set size 10 goes to size 100
• An algorithm that is O(N) would go from time
  proportional to 10 to time proportional to 100, a
  factor of 10 increase.

10
O(N log N)

• Data set size 10 goes to size 100
• An algorithm that is O(N log N) would go from
  time proportional to 10 log 10 to 100 log
  100. Choose base 10, go from 10 1 to 100 2, 10
  to 200 is increase factor of 20.

11
O(N2)

• Data set size 10 goes to size 100
• An algorithm that is O(N2) would go from time
  proportional to 100 to 100100, increase by a
  factor of 100.

12
Compare

• Data set size 10 goes to size 100
• An algorithm that is O(N) would go from time
  proportional to 10 to time proportional to 100, a
  factor of 10 increase.
• An algorithm that is O(N log N) would go from
  time proportional to 10 log 10 to 100 log
  100. Choose base 10, go from 10 1 to 100 2, 10
  to 200 is increase factor of 20.
• An algorithm that is O(N2) would go from time
  proportional to 100 to 100100, increase by a
  factor of 100.

13
Sorting presentations

• heap sort
• quick sort
• merge sort

14
Heapsort

• http//www.ee.ryerson.ca/courses/coe428/sorting/h
  eapsort.html
• Method represent a tree structure using a linear
  area.
• Repeated step make the tree a heap, where heap
  means that the parent node is bigger than either
  of the child nodes
• The root (top) node will be the maximum, so swap
  it to the last position and then repeat for all
  up to but not including that last position.
• That swap probably means that it is no longer a
  heap, so heapify again and keep going.

15
Quicksort

• http//www.sorting-algorithms.com/quick-sort
• http//en.wikipedia.org/wiki/Quicksort
• Method choose a value, called the pivot and then
  compare and divide the whole set into less than
  pivot and greater than pivot.
• Repeat for each part.

16
Merge sort

• http//en.wikipedia.org/wiki/Merge_sort
• http//www.sorting-algorithms.com/merge-sort
• Method divide into sub lists (typically 2, but
  if distributed computing is available, would do
  more). Sort each list, possibly using merge
  sort.Combine by advancing (my word) the one that
  is the least.

17
Sorting

• Stability does sort preserve original order for
  elements that are in order? This clearly is
  required for sorting on multiple fields.
• Is method easily (productively) parallelizable?
• Space can it be done in-place?

18
Binary search

• Have a sorted set (key data) with M entries.
• Have new data and want to find it in the set and
  replace OR insert in correct position.
• How to do it?
• compare key to item in the middle. If a match,
  okay. If not, if new key is less, repeat for the
  upper half, else repeat for the lower half.
• How many compares?

19
Answer

• Say M 2N, then answer is N.
• Put another way, answer is ceiling(log2(M))
• Sobinary search is O(log2(M))
• This is less than linear If size of set goes up
  factor of 4, then time for search goes up by
  factor of 2.

20
Other measures

• Space requirements
• Note some sorts are done 'in place'
• Focus on specific operations, such as file reads
  or writes
• ?

21
Homework

• Extra credit present another (very different)
  sort or search or
• Postings
• Presentations