Sorting

1
Sorting

• CS-240 CS-341
• Dick Steflik

2
Exchange Sorting

• Method make n-1 passes across the data, on each
  pass compare adjacent items, swapping as
  necessary (n-1 compares)
• O(n2)

3
Exchange Sorting
Basic Algorithm ( no improvements)
int ExchangeSort ( int num , const int
numel ) int i,j,k, moves 0 for (
i 0 i lt (numel - 1) i ) for (
j 0 j lt (numel - 1) j )
if ( num j lt num j-1 )
temp num j
num j num j - 1
num j - 1
temp moves
return moves
4
Exchange Sort
Improvement 1 dont revist places that are
already in the correct place
int ExchangeSort ( int num , const int
numel ) int i,j,k, moves 0 for (
i 0 i lt (numel - 1) i ) for (
j i j lt numel j ) if (
num j lt num j-1 )
temp num j
num j num j - 1
num j - 1 temp
moves
return moves
5
Exchange Sort
int ExchangeSort ( int num , const int
numel ) int i,j,k, swap,moves 0 swap
1 for ( i 0 i lt (numel - 1) i )
if ( swap ! 0 ) swap
0 for ( j 0 j lt numel
j ) if ( num j lt
num j-1 )
temp num j
num j num j -
1 num j -
1 temp
moves swap
1
return moves
// Improvement 2 // if a pass through the
inner loop occurs // that doesnt product a
swap, youre done. // both of the improvements
improve the // overall performance but to not
improve // O(n2)
6
Exchange Sort
3
7
5
2
4
compare 3 and 7 7 is gt 3 so advance
3
5
7
2
4
compare 7 and 5, 7 gt 5 so swap them
3
5
2
7
4
compare 7 and 2, 7 gt4 so swap them
3
5
2
4
7
compare 7 and 4, 7 gt4 so swap them
End of pass 1 notice that 7 is in the right place
7
Exchange Sort - pass 2
3
5
2
4
7
3
5
2
4
7
compare 3 and 5 3 is gt5 so advance
3
2
5
4
7
compare 5 and 2, 5 lt 2 so swap them
3
2
4
5
7
compare 5 and 4, 5 gt 4 so swap them
3
2
4
5
7
compare 5 and 7, 5 lt 7 so pass 2 done
End of pass 2 notice that 5 and 7 are in the
right places
8
Exchange Sort - pass 3
3
2
4
5
7
2
3
4
5
7
compare 3 and 2 3 is gt2 so swap them
2
3
4
5
7
compare 3 and 4, 3 lt42 so advance
2
3
4
5
7
compare 4 and 5, 4 lt 5 so advance
2
3
4
5
7
compare 5 and 7, 5 lt 7 so pass 3 done
End of pass 3 notice that 4, 5 and 7 are in the
right places
9
Exchange Sort - pass n-1 (4)
2
3
4
5
7
2
3
4
5
7
compare 2 and 3 2 is lt 3 so advance
2
3
4
5
7
compare 3 and 4, 3 lt4 so advance
2
3
4
5
7
compare 4 and 5, 4 lt 5 so advance
2
3
4
5
7
compare 5 and 7, 5 lt 7 so pass n-1 done
End of pass 4 notice that everything is where it
should be.
10
Exchange Sort Improvements

• on each successive pass, do one less compare,
  because the last item from that pass is in place
• if you ever make a pass in which no swap occurs,
  the sort is complete
• These will both improve performance but Big O
  will remain O(n2)

11
Insertion Sort

• Strategy divide the collection into two lists,
  one listed with one element (sorted) and the
  other with the remaining elements.
• On successive passes take an item from the
  unsorted list and insert it into the sorted list
  so the the sorted list is always sorted
• Do this until the unsorted list is empty

12
Insertion Sort
sorted
unsorted
3
7
5
2
4
take an item from the unsorted list (7) and
insert into the sorted list
sorted
unsorted
3
7
5
2
4
take next item from the unsorted list (5) and
insert into the sorted list
sorted
unsorted
3
5
7
2
4
take next item from the unsorted list (2) and
insert into the sorted list
sorted
unsorted
2
3
5
7
4
take next item from the unsorted list (4) and
insert into the sorted list
sorted
unsorted
2
3
4
5
7
13
Insertion Sort
Void InsertionSort ( int A , int n )
int i , j int temp for ( i i lt n ,
i ) // scan down list looking for
correct place to put new element j
i temp A i
while ( j lt 0 temp lt A j-1 )
// shift the list 1 element to the right to
make room for new element A j
A j1 j--
A j temp
14
Insertion Sort

• Note that each insertion could be O(n-1) and
  there are n-1 insertions being done therefore Big
  O is O(n2)
• This is very much like building an ordered linked
  list except there is more data movement

15
Selection Sort

• Strategy make a pass across the data looking for
  the largest item, swap the largest with the last
  item in the array.
• On successive passes (n-1) assume the array is
  one smaller (the last item is in the correct
  place) and repeat previous step

16
Selection Sort
int SelectionSort ( int num , int numelem
) int i , j , min minidx , temp , moves 0
for ( i 0 i lt numelem i) min
num i minidx i for
( j i 1 j lt numelem j )
min num j minidx j if (
min lt num i ) temp num i
num i min
num minidx moves

17
Selection Sort
biggest
last
3
7
5
2
4
3
4
5
2
7
biggest
last
3
4
5
2
7
3
4
2
5
7
biggest
last
3
4
2
5
7
3
2
4
5
7
3
2
5
7
4
2
3
5
7
4
18
Selection Sort

• Notice that in selection sort, there is the least
  possible data movement
• There are still n-1 compares on sublists that
  become one item smaller on each pass so, Big O is
  still O(n2)
• This method has the best overall performance of
  the O(n2) algorithms because of the limited
  amount of data movement

19
Heap Sort

• A heap is a tree structure in which the key at
  the root is max(keys) and the key of every parent
  is greater than the key of either of its children
• Visualize your array as a complete binary tree
• Arrange the tree into a heap by recursively
  reapplying the heap definition

20
Heap Sort

• repeat the following n-1 times
• swap the root with the last element in the tree
• starting at the root reinforce the heap
  definition
• decrement the index of the last element

21
A Sample Heap
8
Notice largest key is at root
5
7
Notice Keys of children are lt key of parent
3
2
22
Heap Sort
void HeapSort( int A , int n ) // this
constructor turns the array into a max heap
Heap H(A,n) int elt for (int i
n-1 i gt 1 i--)
// delete smallest element from heap and place it
in A i elt H.Hdelete ( )
// Hdelete deletes the largest element and
reenforces the heap property A I
elt
23
Visualize array to sort as tree
0 1 2 3 4
0
3
7
5
2
4
3
2
1
7
5
3
4
2
4
24
Make into a heap
0
0
0
3
7
7
1
7
5
1
3
5
1
4
5
3
4
3
4
3
4
2
4
2
4
2
3
STEP 1
STEP 2
STEP 3
Start at root, compare root to the children, swap
root with largest child
Original Array
Repeat on preorder traversal path. At this point
we have a heap
Note Larger arrays will take more steps
25
Now the sort starts
0
0
0
7
3
5
2
2
2
3
1
4
5
1
4
5
1
4
3
3
4
3
4
3
4
2
3
2
7
2
7
Swap the root and the last node Notice what is
left is no longer a heap
Re heapize it by swaping the root with its
largets child
26
Sorting...
0
0
2
4
2
2
1
1
4
3
2
3
3
4
3
4
5
7
5
7
Reenforce the heap property on the remaining tree
Swap the root and the last node Notice what is
left is no longer a heap
27
Still Sorting...
0
0
0
3
3
2
2
2
2
1
2
4
1
1
2
4
3
4
3
4
3
4
3
4
5
7
5
7
5
7
Swap the root with the last element
Reenforce the heap property if necessary
Swap the root with the last element
28
And what we have left is...
0
2

2
1
3
4
3
4
5
7
0 1 2 3 4
2
3
4
5
7
29
The Merge Principle

• Assume there exist two sorted lists (with queue
  behavior)

7
5
3
1
8
6
4
2
Compare the items at the front of the list and
move the smaller to the back of a third list
7
5
3
1
8
6
4
2
Do it again
7
5
3
1
2
8
6
4
30
The Merge Principle
And again
7
5
3
1
2
8
6
4
And again, and again, and againuntil
7
5
3
1
8
6
4
2
We have a list whose length is the sum of the
lengths and contains all of the elements of both
lists, and this list is also ordered
31
Merge Sort
void MergeSort ( int data , int n )
int n1 , n2 // size of first subarray and
second subarrays respectively if (n gt 1 )
n1 n / 2 n2 n -
n1 MergeSort ( data , n1)
// sort from data0 to datan1-1
MergeSort ((data n1) , n2) // sort from
datan1 to end // merge the two sorted
halves Merge (data , n1 , n2 )

32
Merge Sort
75
17
5
12
19
24
4
27
8
23
14
33
3
11
34
43
Picture the given list as a collection of n, 1
element sorted lists (i.e. 75 is a sorted 1
element list, as is 17. Now merge the adjacent
lists of length 1 into lists of length 2.
17
75
5
12
19
24
4
27
8
23
14
33
3
34
11
43
...now merge the lists of length 2 into lists of
length 4
5
17
12
75
4
19
24
27
23
14
8
34
33
11
3
43
...now merge the lists of length 4 into lists of
length 8
4
5
12
17
19
24
43
34
33
27
23
14
11
8
3
75
...now merge the lists of length 8 into lists of
length 16
3
4
5
8
11
12
14
75
43
34
33
27
23
23
19
17
and there you have it merge sort
33
Quick Sort

• This sorting method by far outshines all of the
  othere for flat out speed
• Big O is log2n
• there are problems, worst case performance is
  when data is already in sorted order or is almost
  in sorted order (well analyze this separately)
• and there are solutions to the problems
• and the is an improvement to make it faster still

34
Quick Sort
Pick the leftmost element as the pivot (23). Now
, start two cursors (one at either end) going
towards the middle and swap values that are gt
pivot (found with left cursor) with values lt
pivot (found with right cursor)
23
17
5
12
19
24
4
27
8
26
14
33
3
11
34
43
swap
23
17
5
12
19
24
4
27
8
26
14
33
3
11
34
43
swap
23
17
5
12
19
24
4
27
8
26
33
3
11
34
14
43
swap
23
17
5
12
19
24
4
27
8
26
33
3
11
34
14
43
swap
Finally, swap the pivot and the value where the
cursors passed each other
23
17
5
12
19
24
4
27
8
26
33
3
11
34
14
43
Note 23 is now in the right place and
everything to its left is lt 23 and everything to
its left is gt 23
35
Quick Sort
Now, repeat the process for the right partition
23
17
5
12
19
24
4
27
8
26
33
3
11
34
14
43
swap
17
5
12
19
4
8
3
11
14
swap
17
5
12
19
4
8
14
3
11
swap
17
5
12
19
4
8
14
3
11
swap
17
5
12
19
4
8
14
3
11
Note the 11 is now in the right place, and the
left partition is all lt pivot and the right
partition is all gt pivot
36
Quick Sort
Now, repeat the process with the right partition
17
5
12
19
4
8
14
3
11
5
4
8
3
Notice that there is nothing to swap , so swap
the pivot and the 4, now the 8 is on the right
place
5
4
8
3
Repeat the process on the leftmost partition again
5
4
3
The 4 is now in the right place and the left and
right partitions are both of size one so they
must also be in the right place
5
4
3
37
Quick Sort
Now that weve exhausted the left partitions,
back up and do the right partition
17
5
12
19
4
8
14
3
11
23
17
12
19
14
swap
17
12
14
swap
Since the left partition is just two items, just
compare and swap if necessary
12
14
17
5
12
19
4
8
14
3
11
23
Now back up and do the remaining right partition
38
Quick Sort
17
5
12
19
4
8
14
3
11
23
24
27
26
33
34
43
swap
24
27
26
33
34
43
swap
24
27
26
33
34
43
swap
24
27
26
33
24
27
26
33
27
26
33
17
5
12
19
4
8
14
3
11
23
24
27
26
33
34
43
All done
39
Quick Sort (worst case)

• If the date us already sorted watch what happens
  to the partitions

17
5
12
19
4
8
14
3
11
23
24
27
26
33
34
43
There is nothing to swap
17
5
12
19
4
8
14
11
23
24
27
26
33
34
43
Again, nothing to swap.. The partitions are
always the maximum size and the performance
degrades to O(n2)
40
Quick Sort (worst case solution)

• The problem comes from the way we picked the
  pivot,
• Pick the pivot a different way
• median-of-three
• instead of always picking the leftmost value of
  the partition use the median of the first, the
  middle and the last value in the partition.

41
Quick Sort Improvement

• Since Quick sort is a recursive algorithm, a lot
  of time gets eaten up in the recursion involved
  with sorting the small partitions
• Solution - use a different algorithm for the
  small partitions
• remember for small collections of data the n2
  algorithms perform better than the log2n
  algorithms
• this is one place where exchange sort excels
• if the partition size is 15 or less use quicksort
  

42
Radix Sort