Sorting Algorithms

1
Sorting Algorithms

2
Sorting means . . .

• The values stored in an array have keys of a type
  for which the relational operators are defined.
  (We also assume unique keys.)
• Sorting rearranges the elements into either
  ascending or descending order within the array.
  (Well use ascending order.)

3
Straight Selection Sort
values 0 1 2
3 4
Divides the array into two parts already
sorted, and not yet sorted. On each pass,
finds the smallest of the unsorted elements, and
swaps it into its correct place, thereby
increasing the number of sorted elements by one.
36 24 10 6 12
4
Selection Sort Pass One
values 0 1 2
3 4
36 24 10 6 12
U N S O R T E D
5
Selection Sort End Pass One
values 0 1 2
3 4
6 24 10 36 12
U N S O R T E D
6
Selection Sort Pass Two
values 0 1 2
3 4
6 24 10 36 12
U N S O R T E D
7
Selection Sort End Pass Two
values 0 1 2
3 4
6 10 24 36 12
U N S O R T E D
8
Selection Sort Pass Three
values 0 1 2
3 4
6 10 24 36 12
U N S O R T E D
9
Selection Sort End Pass Three
values 0 1 2
3 4
S O R T E D
6 10 12 36 24
10
Selection Sort Pass Four
values 0 1 2
3 4
S O R T E D
6 10 12 36 24
11
Selection Sort End Pass Four
values 0 1 2
3 4
6 10 12 24 36
S O R T E D
12
Selection Sort How many comparisons?
values 0 1 2
3 4
6 10 12 24 36
4 compares for values0 3 compares for
values1 2 compares for values2 1 compare
for values3 4 3 2 1
13

• int MinIndex(int Values, int StartIndex, int
  EndIndex)
• // Post Function value index of the smallest
  value in
• // values start . . values end.
• int IndexOfMin
• IndexOfMin StartIndex
• for (int IndexStartIndex1 Index lt
  EndIndex Index)
• if (ValuesIndex lt ValuesIndexOfMin)
• IndexOfMin Index
• return IndexOfMin

13
14

• void Swap(int A, int B)
• int Temp
• Temp A
• A B
• B Temp

14
15

• void SelectionSort(int Values, int NumValues)
• // Post Sorts array values0 . . numValues-1
  into ascending
• // order by key
• int EndIndex
• EndIndex NumValues - 1
• for (int Current 0 Current lt EndIndex
  Current)
• Swap(ValuesCurrent, ValuesMinIndex(Values
  ,Current,EndIndex))

15
16

• include ltiostream.hgt
• include ltfstream.hgt
• const int Max 5
• int MinIndex(int Values, int StartIndex, int
  EndIndex)
• void Swap(int A, int B)
• void SelectionSort(int Values, int NumValues)
• void main()
• int DataMax
• fstream Infile
• int Count
• Infile.open("a\\DataFile.txt", iosin)
• // Initialize Data Array
• for (Count0 CountltMax Count)
• Infile gtgt DataCount
• Infile.close()

16
17

• // Output Data Array before sorting
  
• for (Count0 CountltMax Count)
• coutltltDataCountltltendl
• SelectionSort(Data, Max)
• // Output Data Array after sorting
  
• for (Count0 CountltMax Count)
• coutltltDataCountltltendl
• return

17
18
For selection sort in general

• The number of comparisons when the array contains
  N elements is
• Sum (N-1) (N-2) . . . 2 1

19
Notice that . . .

• Sum (N-1) (N-2) . . . 2
  1
• Sum 1 2 . . .
  (N-2) (N-1)
• 2 Sum N N . . . N
  N
• 2 Sum N (N-1)
• Sum N (N-1)
• 2

20
For selection sort in general

• The number of comparisons when the array contains
  N elements is
• Sum (N-1) (N-2) . . . 2 1
• Sum N (N-1) /2
• Sum .5 N2 - .5 N
• Sum O(N2)

21
Number of Comparisons
22
Order of Magnitude of a Function

• The order of magnitude, or Big-O notation,
• of a function expresses the computing time
• of a problem as the term in a function that
• increases most rapidly relative to the size
• of a problem.

22
23
Names of Orders of Magnitude

• O(1) bounded (by a constant) time
• O(log2N) logarithmic time
• O(N) linear time
• O(Nlog2N) Nlog2N time
• O(N2) quadratic time
• O( 2N ) exponential time

23
24
N log2N Nlog2N N2 2N

25
Big-O Comparison of List Operations

OPERATION UnsortedList
SortedList RetrieveItem O(N) O(N)
linear search O(log2N) binary search
26
Bubble Sort
values 0 1 2
3 4
Compares neighboring pairs of array elements,
starting with the last array element, and swaps
neighbors whenever they are not in correct order.
On each pass, this causes the smallest element
to bubble up to its correct place in the array.
36 24 10 6 12
27

• void BubbleDown ( int Values , int
  StartIndex , int EndIndex )
• // Post Neighboring elements that were out of
  order have been
• // swapped between values StartIndex
  and values EndUndex,
• // beginning at values StartIndex.
• for ( int Index StartIndex Index lt
  EndIndex Index)
• if (Values Index gt Values Index
  1 )
• Swap ( Values Index , Values Index
  1 )
• void BubbleSort ( int Values , int
  NumValues )
• // Post Sorts array values0 . . NumValues-1
  into ascending
• // order by key.
• int Current, EndIndex

27
28

• void BubbleUp ( int Values , int
  StartIndex , int EndIndex )
• // Post Neighboring elements that were out of
  order have been
• // swapped between values StartIndex
  and values EndUndex,
• // beginning at values EndIndex.
• for ( int Index EndIndex Index gt
  StartIndex Index-- )
• if (Values Index lt Values Index -
  1 )
• Swap ( Values Index , Values Index -
  1 )
• void BubbleSort ( int Values , int
  NumValues )
• // Post Sorts array values0 . . NumValues-1
  into ascending
• // order by key.
• int Current 0

28
29

• void BubbleUp ( int Values , int
  StartIndex , int EndIndex, bool Sorted )
• // Post Neighboring elements that were out of
  order have been
• // swapped between values StartIndex
  and values EndUndex,
• // beginning at values EndIndex.
• // Sorted is false if a swap was made
  otherwise, true.
• Sorted true
• for ( int Index EndIndex Index gt
  StartIndex Index-- )
• if (Values Index lt Values Index -
  1 )
• Swap ( Values Index , Values Index -
  1 )
• Sorted false
• void BubbleSort ( int Values , int
  NumValues )
• // Post Sorts array values0 . . NumValues-1
  into ascending
• // order by key. The process stops as
  soon as Values is sorted.

29
30
For bubble sort in general

• The number of comparisons (NumValues - Current -
  1) when the array contains N elements is
• Sum (N-1) (N-2) . . . 2 1
• Sum N (N-1) /2
• Sum .5 N2 - .5 N
• Sum O(N2)

31
For bubble sort with a stop flag

• The number of comparisons when the array contains
  N elements is
• Sum (N-1) (N-2) . . . (N - K)
• Since the BubbleUp function will be called an
  extra time after the Values is sorted and the
  value of Sorted is true.

32
Notice that . . .

• Sum (N-1) (N-2) . . .
  (N-K)
• KN - (1 2 . . . K)
• KN - K(K 1)/2
• (2KN - K2 - K)/2
• O(N2)

33
Insertion Sort
values 0 1 2
3 4
One by one, each as yet unsorted array element is
inserted into its proper place with respect to
the already sorted elements. On each pass, this
causes the number of already sorted elements to
increase by one.
36 24 10 6 12
34
Insertion Sort
Works like someone who inserts one more card at
a time into a hand of cards that are already
sorted. To insert 12, we need to make room for
it by moving first 36 and then 24.
36
12
35
Insertion Sort
Works like someone who inserts one more card at
a time into a hand of cards that are already
sorted. To insert 12, we need to make room for
it by moving first 36 and then 24.
36
12
36
Insertion Sort
Works like someone who inserts one more card at
a time into a hand of cards that are already
sorted. To insert 12, we need to make room for
it by moving first 36 and then 24.
10
6
12
37
Insertion Sort
Works like someone who inserts one more card at
a time into a hand of cards that are already
sorted. To insert 12, we need to make room for
it by moving first 36 and then 24.
12
10
6
38

• void InsertItem ( int Values , int
  StartIndex , int EndIndex )
• // Post Elements between valuesStartIndex and
  
• // valuesEndIndex have been sorted
  into ascending
• // order by key.
• bool Finished false
• int Current EndIndex
• bool MoreToSearch ( Current ! StartIndex )
• while ( MoreToSearch !Finished )
• if (Values Current lt Values
  Current - 1 )
• Swap ( values Current , values
  Current - 1 )
• Current--
• MoreToSearch ( Current ! StartIndex )

38
39

• void InsertionSort ( int Values , int
  NumValues )
• // Post Sorts array values0 . . NumValues-1
  into ascending
• // order by key.
• for ( int Count 0 Count lt NumValues
  Count )
• InsertItem ( Values , 0 , Count )

39
40
For insertion sort in general

• The number of comparisons when the array contains
  N elements is
• Sum 1 2 . . . (N-2) (N-1)
• Sum N (N-1) /2
• Sum .5 N2 - .5 N
• Sum O(N2)

41
Sorting Algorithms and Average Case Number of
Comparisons

• Simple Sorts
• Straight Selection Sort
• Bubble Sort
• Insertion Sort
• More Complex Sorts
• Merge Sort
• Quick Sort
• Heap Sort

O(N2) O(Nlog N)
41
42
Merge Sort Algorithm

• The merge sort algorithm continuously divides a
  list in half and then sorts them as the halves
  combined together.
• To sort a list using merge sort
• If the list contains only one element, it is
  already sorted.
• If the list contains two elements, these elements
  are either in the correct order and the list is
  already sorted, or they are not, so interchange
  them.
• Otherwise, divide the list in half, sort each of
  the two halves, and then merge them.

43
Example

• Sorting a list using merge sort
• Original (Z A C F Q B G K P N D E M H R T)
• Divide (Z A C F Q B G K)(P N D E M H R T)
• Divide (Z A C F)(Q B G K)(P N D E)(M H R T)
• Divide (Z A)(C F)(Q B)(G K)(P N)(D E)(M H)(R T)
• Sort pairs (A Z)(C F)(B Q)(G K)(N P)(D E)(H M)(R
  T)
• Merge (A C F Z)(B G K Q)(D E N P)(H M R T)
• Merge (A B C F G K Q Z)(D E H M N P R T)
• Merge (A B C D E F G H K M N P Q R T Z)

44

• void mergesort(int a, int low, int high)
  
• if(low high) return
• int length high-low1
• int pivot (lowhigh) / 2
• mergesort(a, low, pivot)
• mergesort(a, pivot1, high)
• int working new intlength
• for(int i 0 i lt length i)
• workingi alowi
• int m1 0
• int m2 pivot-low1
• for(int i 0 i lt length i)
• if(m2 lt high-low)
• if(m1 lt pivot-low)
• if(workingm1 gt workingm2)
• ailow workingm2
• else
• ailow workingm1
• else

45
Merge Sort of N elements How many comparisons?
The entire array can be subdivided into halves
only log2N times. Each time it is subdivided,
function Merge is called to re-combine the
halves. Function Merge uses a temporary array to
store the merged elements. Merging is O(N)
because it compares each element in the
subarrays. Copying elements back from the
temporary array to the values array is also
O(N). MERGE SORT IS O(Nlog2N).