# Sorting Algorithms - PowerPoint PPT Presentation

View by Category
Title:

## Sorting Algorithms

Description:

### Sorting Algorithms – PowerPoint PPT presentation

Number of Views:120
Avg rating:3.0/5.0
Slides: 63
Provided by: Wake92
Category:
Tags:
Transcript and Presenter's Notes

Title: Sorting Algorithms

1
Sorting Algorithms
2
Motivation
• Example Phone Book Searching
• If the phone book was in random order, we would
probably never use the phone!
• Potentially, we would need to look at every entry
in the book
• Lets say ½ second per entry
• There are 72,000 households in Ilam
• 35,000 seconds 10hrs to find a phone number!
• Next time make a call, could potentially take
another 10 hours
• We might get lucky and the number we are looking
for is the 1st number ½ second total
• On average though, would be much more time
expensive than the phone call itself (average

3
Motivation
• Because we know the phone book is sorted
• Can use heuristics to facilitate our search
• Jump directly to the letter of the alphabet we
are interested in using
• Scan quickly to find the first two letters that
are really close to the name we are interested in
• Flip whole pages at a time if not close enough
• Every time we pick up the phone book, know it
will take a short time to find what we are
looking for

4
The Big Idea
• Take a set of N randomly ordered pieces of data
aj and rearrange data such that for all j (j gt
0 and j lt N), R holds, for relational operator R
• a0 R a1 R a2 R aj R aN-1 R aN
• If R is lt, we are doing an ascending sort Each
consecutive item in the list is going to be
larger than the previous
• If R is gt, we are doing a descending sort
Items get smaller as move down the list

5
Sorting Algorithms
• What does sorting really require?
• Really a repeated two step process
• Compare pieces of data at different positions
• Swap the data at those positions until order is
correct
• For large groups of data,
• Implementation by a computer program would be
very useful
• Computers can do the data management (compare and
swap) faster than people.

6
Example
20
3
18
9
5
7
Selection Sort
• void selectionSort(int a, int size)
• for (int k 0 k lt size-1 k)
• int index mininumIndex(a, k, size)
• swap(ak,aindex)
• int minimumIndex(int a, int first, int last)
• int minIndex first
• for (int j first 1 j lt last j)
• if (aj lt aminIndex) minIndex j
• return minIndex

8
Selection Sort
• What is selection sort doing?
• Repeatedly
• Finding smallest element by searching through
list
• Inserting at front of list
• Moving front of list forward by 1

9
Selection Sort Step Through
20
3
18
9
5
minIndex(a, 0, 5) ? 1 swap (a0,a1)
10
Order From Previous
Find minIndex (a, 1, 5) 4
Find minIndex (a, 2, 5) 3
11
Find minIndex (a, 3, 5) 3
K 4 size-1 Done!
12
Cost of Selection Sort
• void selectionSort(int a, int size)
• for (int k 0 k lt size-1 k)
• int index mininumIndex(a, k, size)
• swap(ak,aindex)
• int minimumIndex(int a, int first, int last)
• int minIndex first
• for (int j first 1 j lt last j)
• if (aj lt aminIndex) minIndex j
• return minIndex

13
Cost of Selection Sort
• How many times through outer loop?
• Iteration is for k 0 to lt (N-1) gt N-1 times
• How many comparisons in minIndex?
• Depends on outer loop Consider 5 elements
• K 0 j 1,2,3,4
• K 1 j 2, 3, 4
• K 2 j 3, 4
• K 3 j 4
• Total comparisons is equal to 4 3 2 1,
which is N-1 N-2 N-3 1
• What is that sum?

14
Cost of Selection Sort
• (N-1) (N-2) (N-3) 3 2 1
• (N-1) 1 (N-2) 2 (N-3) 3
• N N N gt repeated addition of N
• There were n-1 total starting objects to add, we
grouped every 2 together approximately N/2
• gt Approximately N N/2 O(N2) comparisons

15
Insertion Sort
• void insertionSort(int a, int size)
• for (int k 1 k lt size k)
• int temp ak
• int position k
• while (position gt 0 aposition-1 gt temp)
• aposition aposition-1
• position--
• aposition temp

16
Insertion Sort
• List of size 1 (first element) is already sorted
• Repeatedly
• Chooses new item to place in list (ak)
• Starting at back of the list, if new item is less
than item at current position, shift current data
right by 1.
• Repeat shifting until new item is not less than
thing in front of it.
• Insert the new item

17
Insertion Sort
• Insertion Sort is the classic card playing sort.
• Pick up cards one at a time.
• When select new card, slide it in at the
appropriate point by shifting everything right.
• Easy for humans because shifts are free, where
for computer have to do data movement.

18
Insertion Sort Step Through
20
3
18
9
5
A1
A2
A3
A4
A0
Move 3 left until hits something smaller
20
3
18
9
5
A2
A3
A4
A0
A1
19
Move 3 left until hits something smaller Now
two sorted
18
9
5
3
20
A2
A3
A4
A0
A1
Move 18 left until hits something smaller
20
18
3
9
5
A3
A4
A0
A1
A2
20
Move 18 left until hits something smaller Now
three sorted
9
5
3
18
20
A3
A4
A0
A1
A2
Move 9 left until hits something smaller
3
20
9
18
5
A4
A0
A1
A2
A3
21
Move 9 left until hits something smaller Now
four sorted
3
9
18
20
5
A4
A0
A1
A2
A3
Move 5 left until hits something smaller
3
9
18
20
5
A0
A1
A2
A3
A4
22
Move 5 left until hits something smaller Now
all five sorted Done
3
9
18
20
5
A0
A1
A2
A3
A4
23
Cost of Insertion Sort
• void insertionSort(int a, int size)
• for (int k 1 k lt size k)
• int temp ak
• int position k
• while (position gt 0 aposition-1 gt temp)
• aposition aposition-1
• position--
• aposition temp

24
Cost of Insertion Sort
• Outer loop
• K 1 to lt size 1,2,3,4 gt N-1
• Inner loop
• Worst case Compare against all items in list
• Inserting new smallest thing
• K 1, 1 step (position k 1, while position gt
0)
• K 2, 2 steps position 2,1
• K 3, 3 steps position 3,2,1
• K 4, 4 steps position 4,3,2,1
• Again, worst case total comparisons is equal to
sum of I from 1 to N-1, which is O(N2)

25
Cost of Swaps
• Selection Sort
• void selectionSort(int a, int size)
• for (int k 0 k lt size-1 k)
• int index mininumIndex(a, k, size)
• swap(ak,aindex)
• One swap each time, for O(N) swaps

26
Cost of Swaps
• Insertion Sort
• void insertionSort(int a, int size)
• for (int k 1 k lt size k)
• int temp ak
• int position k
• while (position gt 0 aposition-1 gt temp)
• aposition aposition-1
• position--
• aposition temp
• Do a shift almost every time do compare, so O(n2)
shifts
• Shifts are faster than swaps (1 step vs 3 steps)
• Are we doing few enough of them to make up the
difference?

27
Another Issue - Memory
• Space requirements for each sort?
• All of these sorts require the space to hold the
array - O(N)
• Require temp variable for swaps
• Require a handful of counters
• Can all be done in place, so equivalent in
terms of memory costs
• Not all sorts can be done in place though!

28
Which O(n2) Sort to Use?
• Insertion sort is the winner
• Worst case requires all comparisons
• Most cases dont (jump out of while loop early)
• Selection use for loops, go all the way through
each time

29
• Given random data, when is it more efficient to
• Just search versus
• Insertion Sort and search
• Assume Z searches
• Search on random data Z O(n)
• Sort and binary search O(n2) Z log2n

30
• Z n lt n2 (Z log2n)
• Z n Z log2n lt n2
• Z (n-log2n) lt n2
• Z lt n2/(n-log2n)
• For large n, log2n is dwarfed by n in (n-log2n)
• Z lt n2/n
• Z lt n (approximately)

31
Improving Sorts
• Better sorting algorithms rely on divide and
conquer (recursion)
• Find an efficient technique for splitting data
• Sort the splits separately
• Find an efficient technique for merging the data
• Well see two examples
• One does most of its work splitting
• One does most of its work merging

32
Quicksort
• General Quicksort Algorithm
• Select an element from the array to be the pivot
• Rearrange the elements of the array into a left
and right subarray
• All values in the left subarray are lt pivot
• All values in the right subarray are gt pivot
• Independently sort the subarrays
• No merging required, as left and right are
independent problems Parallelism?!?

33
Quicksort
• void quicksort(int arrayOfInts, int first, int
last)
• int pivot
• if (first lt last)
• pivot partition(arrayOfInts, first,
last)
• quicksort(arrayOfInts,first,pivot-1)
• quicksort(arrayOfInts,pivot1,last)

34
Quicksort
• int partition(int arrayOfInts, int first, int
last)
• int temp
• int p first // set pivot first index
• for (int k first1 k lt last k) // for
every other indx
• if (arrayOfIntsk lt arrayOfIntsfirst) //
if data is smaller
• p p 1 // update final pivot location
• swap(arrayOfIntsk, arrayOfIntsp)
• swap(arrayOfIntsp, arrayOfIntsfirst)
• return p

35
Partition Step Through
partition(cards, 0, 4) P 0 K 1 P 1 K
3 cards1 lt cards0 ? No cards3 lt
cards0? Yes P 2 P 0 K 2 temp
cards3 cards2 lt cards0 ? Yes cards3
cards2 P 1 cards2 cards3 temp
cards2 P 2 K 4 cards2
cards1 cards4 lt cards0? No cards1
temp temp cards2, cards2
cardsfirst cardsfirst temp, return p
2
36
Complexity of Quicksort
• Worst case is O(n2)
• What does worst case correspond to?
• Already sorted or near sorted
• Partitioning leaves heavily unbalanced subarrays
• On average is O(n log2n), and it is average a lot
of the time.

37
Complexity of Quicksort
• Recurrence Relation Average Case
• 2 sub problems
• ½ size (if good pivot)
• Partition is O(n)
• a 2 b 2 k 1
• 2 21 Master Theorem O(nlog2n)

38
Complexity of Quicksort
• Recurrence Relation Worst Case
• Partition separates into (n-1) and (1)
• Cant use master theorem
• b (subproblem size) changes
• n-1/n n-2/n-1 n-3/n-2
• Note that sum of partition work
• n (n-1) (n-2) (n-3)
• Sum(1,N) N(N1)/2 O(N2)

39
Complexity of Quicksort
• Requires stack space to implement recursion
• Worst case O(n) stack space
• If pivot breaks into 1 element and n-1 element
subarrays
• Average case O(log n)
• Pivot splits evenly

40
MergeSort
• General Mergesort Algorithm
• Recursively split subarrays in half
• Merge sorted subarrays
• Splitting is first in recursive call, so
continues until have one item subarrays
• One item subarrays are by definition sorted
• Merge recombines subarrays so result is sorted
• 11 item subarrays gt 2 item subarrays
• 22 item subarrays gt 4 item subarrays
• Use fact that subarrays are sorted to simplify
merge algorithm

41
MergeSort
• void mergesort(int array, int tempArray, int
low, int high, int size)
• if (low lt high)
• int middle (low high) / 2
• mergesort(array,tempArray,low,middle,
size)
• mergesort(array,tempArray,middle1, high,
size)
• merge(array,tempArray,low,middle,high,
size)

42
MergeSort
• void merge(int array, int tempArray, int low,
int middle, int high, int size)
• int i, j, k
• for (i low i lt high i) tempArrayi
arrayi // copy into temp array
• i low j middle1 k low
• while ((i lt middle) (j lt high)) //
merge
• if (tempArrayi lt tempArrayj) // if
lhs item is smaller
• arrayk tempArrayi // put in final
array, increment
• else // final array position, lhs index
• arrayk tempArrayj // else put rhs
item in final array
• // increment final array position
• // rhs index
• while (i lt middle) // one of the two will
run out
• arrayk tempArrayi // copy the
rest of the data
• // only need to copy if in lhs
array // rhs array already in right place

43
MergeSort Example
20
3
18
9
5
Recursively Split
20
3
18
9
5
44
MergeSort Example
20
3
18
9
5
Recursively Split
9
20
3
18
5
45
MergeSort Example
20
3
18
9
5
Merge
46
Merge Sort Example
2 cards Not very interesting Think of as swap
20
3
20
3
Temp Array
Array
Tempi lt Tempj Yes
18
20
3
3
k
j
i
47
MergeSort Example
Temp Array
Array
Tempi lt Tempj No
18
20
3
3
18
j
k
i
Update J, K by 1 gt Hit Limit of Internal While
Loop, as J gt High Now Copy until I gt Middle
Array
20
k
48
MergeSort Example
2 Card Swap
9
5
5
9
20
18
3
Final after merging above sets
i0,j3
49
Complexity of MergeSort
• Recurrence relation
• 2 subproblems
• ½ size
• Merging is O(n) for any subproblem
• Always moving forwards in the array
• a 2 b 2 k 1
• 2 21 Master Theorem O(n log2n)
• Always O(n log2n) in both average and worst case
• Doesnt rely on quality of pivot choice

50
Space Complexity of Mergesort
• Need an additional O(n) temporary array
• Number of recursive calls
• Always O(log2n)

51
• When it is more useful to
• Just search
• Quicksort or Mergesort and search
• Assume Z searches
• Search on random data Z O(n)
• Fast Sort and binary search
• O(nlog2n) Z log2n

52
• Z n lt nlog2n Zlog2n
• Z(n - log2n) lt n log2n
• Z lt (n log2n) / (n-log2n)
• Z lt (n log2n) / n Approximation
• Z lt log2n Approximation
• Where as before, had to do N searches to make up
for cost of sorting, now only do log2N
• 1,000,000 items 19 searches, instead of
1,000,000

53
How Fast?
• Without specific details of what sorting, O(n
log2n) is the maximum speed sort possible.
• Only available operations Compare, Swap
• Proof Decision Tree describes how sort
operates
• Every vertex represents a comparison, every
branch a result
• Moving down tree Tracing a possible run
through the algorithm

54
How Fast?
K1 lt K2
1,2,3
Yes
No
K2 lt K3
K1 lt K3
2,1,3
1,2,3
Yes
No
Yes
No
2,3,1
1,2,3
K2 lt K3
stop
2,1,3
stop
K1 lt K3
1,3,2
Yes
Yes
No
No
stop
stop
stop
stop
1,3,2
3,1,2
2,3,1
3,2,1
55
How Fast?
• There are n! possible stop nodes effectively
all permutations of the n numbers in the array.
• Thus any decision tree representing a sorting
algorithm must have n! leaves
• The height of a this type of tree (a binary tree)
is correlated with number of leaves
• Height k 2(k-1) leaves
• Must be at least log2n! 1 height

56
How Fast?
• Path from top to bottom of tree trace of a run
of the algorithm
• Need to prove that (log2n!) is lower bounded by
(n log2n)
• n! (n)(n-1)(n-2)(n-3) (3)(2)(1)
• gt (n)(n-1)(n-2)(n-3) ceil(n/2) // doing
fewer multiplies
• gt ceil(n/2) (ciel(n/2)) // doing multiplies of
bigger things
• gt approximately (n/2)(n/2)
• log 2 n! gt log 2 (n/2)(n/2)
• log 2 n! gt (n/2) log 2 (n/2) //exponentiation in
logs multiplication out front
• log 2 n! gt (n/2)(log2n log2 2) // division in
logs subtraction
• log 2 n! gt (n/2)(log2n 1)
• log 2 n! gt (n/2)(log2n) (n/2)
• log 2 n! gt (1/2) nlog2n n
• log 2 n! O(n log2n)

57
How Fast?
• Lower bound of n2? No
• Would Need to prove log2 n! gt n2
• Meaning we would need to prove n! gt 2(n2)

58
Stable Sorts
• Stable sort
• A sort that preserves the input ordering of
elements that are equal
• Why important?
• Maintenance of pre-specified ordering /
properties
• An example Preservation of a previous sort on
another key

59
Stable Sorts
• Which sorts have we seen that are stable?
• Insertion Sort
• Picking up in order from extras pile
• Only move past if you are greater than item in
front of you
• Mergesort
• No swapping
• Fill in result array from temp array left to
right
• Merging Two equal things one on the left
stays on the left
• Unstable
• Selection Sort - Why?
• Quick Sort - Why?

60
Selection Sort Unstable
• void selectionSort(int a, int size)
• for (int k 0 k lt size-1 k)
• int index mininumIndex(a, k, size)
• swap(ak,aindex)
• int minimumIndex(int a, int first, int last)
• int minIndex first
• for (int j first 1 j lt last j)
• if (aj lt aminIndex) minIndex j
• return minIndex

Unstable because swapping may rearrange order of
original items (Sort doesnt care what it picks
up and moves)
61
Selection Sort - Unstable
2a
3a
3b
2b
Do Swap to Put 2 of Spades in Place for Value
Sort Ruins 3s Type Sort
2a
3a
3b
2b
62
Quicksort
• int partition(int arrayOfInts, int first, int
last)
• int temp
• int p first // set pivot first index
• for (int k first1 k lt last k) //
for every other index
• if (arrayOfIntsk lt arrayOfIntsfirst)
// if data on is smaller
• p p 1 // update final pivot location
• temp arrayOfIntsk // rearrange data
• arrayOfIntsk arrayOfIntsp
• arrayOfIntsp temp
• temp arrayOfIntsp // put pivot in right
spot
• arrayOfIntsp arrayOfIntsfirst
• arrayOfIntsfirst temp
• return p

Possibility to put pivot/item where pivot belongs
out of order w/ respect to previous sort