Sorting Algorithms and Average Case Time Complexity

1
Sorting Algorithms and Average Case Time
Complexity

• Simple Sorts O(n2)
• Insertion Sort
• Selection Sort
• Bubble Sort
• More Complex Sorts O(nlgn)
• Heap Sort
• Quick Sort
• Merge Sort

2
Heap Sort

• Remember the heap data structure . . .
• Binary heap
• a binary tree that is
• Complete
• satisfies the heap property

3
Heap Sort (contd)

• Given an array of integers
• First, use MakeHeap to convert the array into a
  max-heap.
• Then, repeat the following steps until there are
  no more unsorted elements
• Take the root (maximum) element off the heap by
  swapping it into its correct place in the array
  at the end of the unsorted elements
• Heapify the remaining unsorted elements (This
  puts the next-largest element into the root
  position)

4
Heap Sort (contd)

• HeapSort(A, n)
• MakeHeap(A)
• for i n downto 2
• exchange A1 with Ai
• n n-1
• Heapify(A,1)

5
Heapsort Example

• 2 8 6 1 10 15 12 11

6
Heap Sort (contd)

• Time complexity?
• First, MakeHeap(A)
• Easy analysis O(nlgn) since Heapify is O(lgn)
• More exact analysis O(n)
• Then
• n-1 swaps O(n)
• n-1 calls to Heapify O(nlgn)
• ? O(n) O(n) O(nlgn) O(nlgn)

7
Other Sorting Ideas

• Divide and Conquer!

8
Quicksort

• Recursive in nature.
• First Partition the array, and recursively
  quicksort the subarray separately until the whole
  array is sorted
• This process of partitioning is carried down
  until there are only one-cell arrays that do not
  need to be sorted at all

9
Quicksort Algorithm

• Given an array of integers
• If array only contains one element, return
• Else
• Pick one element to use as the pivot
• Partition elements into 2 subarrays
• Elements less than or equal to the pivot
• Elements greater than or equal to the pivot
• Quicksort the two subarrays

10
Quicksort Example
11
Quicksort Example

• There are a number of ways to pick the pivot
  element. Here, we will use the first element in
  the array

12
Quicksort Partition

• Given a pivot, partition the elements of the
  array such that the resulting array consists of
• One sub-array that contains elements lt pivot
• Another sub-array that contains elements gt pivot
• The sub-arrays are stored in the original array
• Partitioning loops through, swapping elements
  below/above pivot

13
Quicksort Example Partition Result
14
QuickSort
templateltclass Tgt void quicksort(T data, int
first, int last) //partition such that left
lt pivot, right gt pivot int lower first1,
upper last swap(datafirst,data(firstlas
t)/2) T pivot datafirst while
(lower lt upper) //find the position of the
first gt pivot element on the left while
(datalower lt pivot) lower
//find the position of the first lt pivot element
on the right while (pivot lt dataupper)
upper-- if (lower lt upper)
swap(datalower,dataupper--)
else lower swap(dataupper,data
first) if (first lt upper-1)
quicksort (data,first,upper-1) //quicksort
left if (upper1 lt last) quicksort
(data,upper1,last) //quicksort right
15
Quicksort - Recursion

• Recursion Quicksort sub-arrays

16
Quicksort Worst Case Complexity

• Worst case running time?
• Assume first element is chosen as pivot.
• Assume array is already in order

lt
Recursion 1. Partition splits array in two
sub-arrays one sub-array of size 0 the other
sub-array of size n-1 2. Quicksort each
sub-array Depth of recursion tree? O(n)
Number of accesses per partition? O(n)
17
Quicksort Best Case Complexity

• Assume that keys are random, uniformly
  distributed.
• Best case running time O(n log2n)

18
Merge Sort

• Quicksort worst case complexity in the worst
  case is O(n2) because it is difficult to control
  the partitioning process
• The partitioning depends on the choice of pivot,
  no guarantee that partitioning will result in
  arrays of approximately equal size
• Another strategy is to make partitioning as
  simple as possible and concentrate on merging
  sorted (sub)arrays

19
Merge Sort

• Also recursive in nature
• Given an array of integers, apply the
  divide-and-conquer paradigm
• Divide the n-element array into two subarrays of
  n/2 elements each
• Sort the two subarrays recursively using Merge
  Sort
• Merge the two subarrays to produce the sorted
  array
• The recursion stops when the array to be sorted
  has length 1

20
Merge Sort

• Merge Sort
• Divide array into two halves
• Recursively sort each half
• Merge two halves to make sorted whole

21
Merging

• Merging Combine two pre-sorted lists into a
  sorted whole
• How to merge efficiently?
• Use temporary array
• Need index 1 (for array 1), index 2 (for array
  2), 3 (for array tmp)

1
3
19
25
5
7
8
9
29
3
5
7
1
22
Merging

• Merge(A, left, middle, right)
• Create temporary array temp, same size as A
• i1 left
• i2 middle
• i3 0
• while ((i1 ! middle) (i2 ! right))
• if (Ai1 lt Ai2)
• tempi3 Ai1
• else tempi3 Ai2
• copy into temp remaining elements of A
• copy into A the contents of temp

23
Merging

• Time Complexity Analysis
• T(n) c1 if n 1
• T(n) 2 T(n/2) c2n

divide
O(1)
sort
2T(n/2)
merge
O(n)
24
Merge Sort

• MergeSort(A, left, right)
• if (left lt right)
• middle ?(left right)/2?
• MergeSort(A, left, middle)
• MergeSort(A, middle1, right)
• Merge(A, left, middle1, right)