Sorting Algorithms

1
Sorting Algorithms

• Bubble Sort
• Merge Sort
• Quick Sort
• Randomized Quick Sort

2
Overview

• Bubble Sort
• Divide and Conquer
• Merge Sort
• Quick Sort
• Randomization

3
Sorting

• Sorting takes an unordered collection and makes
  it an ordered one.

1 2 3 4 5
6
101
12
5
35
42
77
1 2 3 4 5
6
4
"Bubbling Up" the Largest Element

• Traverse a collection of elements
• Move from the front to the end
• Bubble the largest value to the end using
  pair-wise comparisons and swapping

1 2 3 4 5
6
12
101
5
35
42
77
5
"Bubbling Up" the Largest Element

• Traverse a collection of elements
• Move from the front to the end
• Bubble the largest value to the end using
  pair-wise comparisons and swapping

1 2 3 4 5
6
Swap
12
101
5
35
42
77
6
"Bubbling Up" the Largest Element

• Traverse a collection of elements
• Move from the front to the end
• Bubble the largest value to the end using
  pair-wise comparisons and swapping

1 2 3 4 5
6
Swap
12
101
5
35
77
42
7
"Bubbling Up" the Largest Element

• Traverse a collection of elements
• Move from the front to the end
• Bubble the largest value to the end using
  pair-wise comparisons and swapping

1 2 3 4 5
6
Swap
12
101
5
77
35
42
8
"Bubbling Up" the Largest Element

• Traverse a collection of elements
• Move from the front to the end
• Bubble the largest value to the end using
  pair-wise comparisons and swapping

1 2 3 4 5
6
77
101
5
12
35
42
No need to swap
9
"Bubbling Up" the Largest Element

• Traverse a collection of elements
• Move from the front to the end
• Bubble the largest value to the end using
  pair-wise comparisons and swapping

1 2 3 4 5
6
Swap
77
101
5
12
35
42
10
"Bubbling Up" the Largest Element

• Traverse a collection of elements
• Move from the front to the end
• Bubble the largest value to the end using
  pair-wise comparisons and swapping

1 2 3 4 5
6
101
77
5
12
35
42
Largest value correctly placed
11
The Bubble Up Algorithm

• index
• last_compare_at
• loop
• exitif(index last_compare_at)
• if(Aindex Aindex 1) then
• Swap(Aindex, Aindex 1)
• endif
• index
• endloop

12
No, Swap isnt built in.

• Procedure Swap(a, b isoftype in/out Num)
• t isoftype Num
• t
• a
• b
• endprocedure // Swap

13
Items of Interest

• Notice that only the largest value is correctly
  placed
• All other values are still out of order
• So we need to repeat this process

1 2 3 4 5
6
101
77
5
12
35
42
Largest value correctly placed
14
Repeat Bubble Up How Many Times?

• If we have N elements
• And if each time we bubble an element, we place
  it in its correct location
• Then we repeat the bubble up process N 1
  times.
• This guarantees well correctly place all N
  elements.

15
Bubbling All the Elements
16
Reducing the Number of Comparisons
17
Reducing the Number of Comparisons

• On the Nth bubble up, we only need to do MAX-N
  comparisons.
• For example
• This is the 4th bubble up
• MAX is 6
• Thus we have 2 comparisons to do

18
Putting It All Together

• N is // Size of Array
• Arr_Type definesa Array1..N of Num
• Procedure Swap(n1, n2 isoftype Num)
• temp isoftype Num
• temp
• n1
• n2
• endprocedure // Swap

19
The Truth

• NOBODY EVER uses Bubble Sort (except for Ezra)
• NOBODY
• NOT EVER
• Because it is EXTREMELY INEFFICIENT

20
This is how it goes
21
Comparison (semi-log y axis)
22
Lets forget about Bubble Sort
23
Divide and Conquer

• Base Case, solve the problem directly if it is
  small enough
• Divide the problem into two or more similar and
  smaller subproblems
• Recursively solve the subproblems
• Combine solutions to the subproblems

24
Divide and Conquer - Sort

• Problem
• Input Aleft..right unsorted array of
  integers
• Output Aleft..right sorted in non-decreasing
  order

25
Divide and Conquer - Sort

• Base case
• at most one element (left right), return
• Divide A into two subarrays FirstPart,
  SecondPart
• Two Subproblems
• sort the FirstPart
• sort the SecondPart
• Recursively
• sort FirstPart
• sort SecondPart
• Combine sorted FirstPart and sorted SecondPart

26
This reminds me of
Thats easy. Mariah Carey. As a singing star,
Ms. Carey has perfected the wax-on
wave motion--a clockwise sweep of her hand used
to emphasize lyrics.
The Maria Wax-on Angle (q,t)
27
How To Remember Merge Sort?
(q,t)
Just as Mariah recursively moves her hands into
smaller circles, so too does merge sort
recursively split an array into smaller segments.

We need two such recursions, one for each half of
the split array.
28
Overview

• Divide and Conquer
• Merge Sort
• Quick Sort
• Randomization

29
Merge Sort Idea
Divide into two halves
A
FirstPart
SecondPart
Recursively sort
SecondPart
FirstPart
A is sorted!
30
Merge Sort Algorithm
Merge-Sort (A, left, right) if left
right return else middle ? b(leftright)/2? Me
rge-Sort(A, left, middle) Merge-Sort(A, middle1,
right) Merge(A, left, middle, right)
Recursive Call
31
Merge-Sort Merge
Sorted
A
merge
Sorted SecondPart
Sorted FirstPart
A
Aright
Aleft
Amiddle
32
Merge-Sort Merge Example
A
33
Merge-Sort Merge Example
A
3
5
15
28
30
6
10
14
1
k0
R
L
3
15
28
30
2
3
7
8
6
10
14
22
1
4
5
6
i0
j0
34
Merge-Sort Merge Example
A
2
1
5
15
28
30
6
10
14
k1
R
L
3
5
15
28
2
3
7
8
6
10
14
22
1
4
5
6
i0
j1
35
Merge-Sort Merge Example
A
3
1
2
15
28
30
6
10
14
k2
R
L
2
3
7
8
6
10
14
22
1
4
5
6
i1
j1
36
Merge-Sort Merge Example
A
1
2
3
6
10
14
4
k3
R
L
2
3
7
8
6
10
14
22
1
4
5
6
j1
i2
37
Merge-Sort Merge Example
A
1
2
3
4
6
10
14
5
k4
R
L
2
3
7
8
6
10
14
22
1
4
5
6
i2
j2
38
Merge-Sort Merge Example
A
1
2
3
4
5
6
10
14
6
k5
R
L
2
3
7
8
6
10
14
22
1
4
5
6
i2
j3
39
Merge-Sort Merge Example
A
7
1
2
3
4
5
6
14
k6
R
L
2
3
7
8
6
10
14
22
1
4
5
6
i2
j4
40
Merge-Sort Merge Example
A
8
1
2
3
4
5
6
7
14
k7
R
L
3
5
15
28
2
3
7
8
6
10
14
22
1
4
5
6
i3
j4
41
Merge-Sort Merge Example
A
1
2
3
4
5
6
7
8
k8
R
L
3
5
15
28
2
3
7
8
6
10
14
22
1
4
5
6
j4
i4
42

• Merge(A, left, middle, right)
• n1 ? middle left 1
• n2 ? right middle
• create array Ln1, Rn2
• for i ? 0 to n1-1 do Li ? Aleft i
• for j ? 0 to n2-1 do Rj ? Amiddlej
• k ? i ? j ? 0
• while i
• if Li
• Ak ? Li
• else
• Ak ? Rj
• while i
• Ak ? Li
• while j
• Ak ? Rj

n n1n2 Space n Time cn for some constant c
43
Merge-Sort(A, 0, 7)
Divide
A
3 7 5 1
6 2 8 4
6 2 8 4 3
7 5 1
44
Merge-Sort(A, 0, 7)
, divide
Merge-Sort(A, 0, 3)
A
3 7 5 1
8 4
6 2 8 4
6 2
45
Merge-Sort(A, 0, 7)
, divide
Merge-Sort(A, 0, 1)
A
3 7 5 1
8 4
2
6 2
6
46
Merge-Sort(A, 0, 7)
, base case
Merge-Sort(A, 0, 0)
A
3 7 5 1
8 4
2
6
47
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 0), return
A
3 7 5 1
8 4
2
6
48
Merge-Sort(A, 0, 7)
, base case
Merge-Sort(A, 1, 1)
A
3 7 5 1
8 4
6
2
49
Merge-Sort(A, 0, 7)
Merge-Sort(A, 1, 1), return
A
3 7 5 1
8 4
2
6
50
Merge-Sort(A, 0, 7)
Merge(A, 0, 0, 1)
A
3 7 5 1
8 4
2 6
51
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 1), return
A
3 7 5 1
8 4
2 6
52
Merge-Sort(A, 0, 7)
, divide
Merge-Sort(A, 2, 3)
A
3 7 5 1
2 6
4
8
8 4
53
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), base case
A
3 7 5 1
2 6
4
8
54
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 2), return
A
3 7 5 1
2 6
4
8
55
Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), base case
A
2 6
8
4
56
Merge-Sort(A, 0, 7)
Merge-Sort(A, 3, 3), return
A
3 7 5 1
2 6
4
8
57
Merge-Sort(A, 0, 7)
Merge(A, 2, 2, 3)
A
3 7 5 1
2 6
4 8
58
Merge-Sort(A, 0, 7)
Merge-Sort(A, 2, 3), return
A
3 7 5 1
4 8
2 6
59
Merge-Sort(A, 0, 7)
Merge(A, 0, 1, 3)
A
3 7 5 1
2 4 6 8
60
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 3), return
A
3 7 5 1
2 4 6 8
61
Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7)
A
2 4 6 8
3 7 5 1
62
Merge-Sort(A, 0, 7)
Merge (A, 4, 5, 7)
A
2 4 6 8
1 3 5 7
63
Merge-Sort(A, 0, 7)
Merge-Sort(A, 4, 7), return
A
2 4 6 8
1 3 5 7
64
Merge-Sort(A, 0, 7)
Merge-Sort(A, 0, 7), done!
Merge(A, 0, 3, 7)
A
1 2 3 4 5
6 7 8
65
Merge-Sort Analysis
cn
n
2 cn/2 cn
n/2
n/2
log n levels
4 cn/4 cn
n/4
n/4
n/4
n/4
n/2 2c cn
2
2
2
Total cn log n

• Total running time ?(nlogn)
• Total Space ? (n)

66
Merge-Sort Summary

• Approach divide and conquer
• Time
• Most of the work is in the merging
• Total time ?(n log n)
• Space
• ?(n), more space than other sorts.

67
Overview

• Divide and Conquer
• Merge Sort
• Quick Sort

68
Quick Sort

• Divide
• Pick any element p as the pivot, e.g, the first
  element
• Partition the remaining elements into
• FirstPart, which contains all elements
• SecondPart, which contains all elements p
• Recursively sort the FirstPart and SecondPart
• Combine no work is necessary since sorting
• is done in place

69
Quick Sort
A
pivot
FirstPart
SecondPart
x Sorted
70
Quick Sort

• Quick-Sort(A, left, right)
• if left right return
• else
• middle ? Partition(A, left, right)
• Quick-Sort(A, left, middle1 )
• Quick-Sort(A, middle1, right)
• end if

71
Partition
A
A
A
p
72
Partition Example
A
4
8
6
3
5
1
7
2
73
Partition Example
i0
A
4
8
6
3
5
1
7
2
j1
74
Partition Example
i0
A
4
8
6
3
5
1
7
2
8
j1
75
Partition Example
i0
A
4
8
6
3
5
1
7
2
6
j2
76
Partition Example
i0
A
4
8
6
3
5
1
7
2
3
j3
77
Partition Example
A
4
3
6
8
5
1
7
2
5
j4
78
Partition Example
A
4
3
6
8
5
1
7
2
1
j5
79
Partition Example
A
4
3
6
8
5
1
7
2
j5
80
Partition Example
A
7
4
3
8
5
7
2
1
6
j6
81
Partition Example
A
2
2
8
4
3
8
5
7
2
1
6
j7
82
Partition Example
A
4
3
2
6
7
8
1
5
j8
83
Partition Example
A
4
4
1
6
7
8
2
5
2
3
84
Partition Example
pivot in correct position
A
4
2
3
6
7
8
1
5
85

• Partition(A, left, right)
• x ? Aleft
• i ? left
• for j ? left1 to right
• if Aj
• i ? i 1
• swap(Ai, Aj)
• end if
• end for j
• swap(Ai, Aleft)
• return i

n right left 1 Time cn for some
constant c Space constant
86
Quick-Sort(A, 0, 7)
Partition
A
4 8 6 3 5
1 7 2
87
Quick-Sort(A, 0, 7)
, partition
Quick-Sort(A, 0, 2)
A
5 6 7 8
4
2 3 1
88
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 0)
, base case
, return
5 6 7 8
4
3
1
2
1
89
Quick-Sort(A, 0, 7)
Quick-Sort(A, 1, 1)
, base case
5 6 7 8
4
1
2
90
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 2), return
Quick-Sort(A, 2, 2), return
5 6 7 8
4
91
Quick-Sort(A, 0, 7)
Quick-Sort(A, 4, 7)
, partition
Quick-Sort(A, 2, 2), return
4
92
Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7)
, partition
4
5
6
6 7 8
93
Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7)
, partition
4
5
6
7 8
8
7
94
Quick-Sort(A, 0, 7)
, return
, base case
Quick-Sort(A, 7, 7)
4
5
6
7
8
8
95
Quick-Sort(A, 0, 7)
, return
Quick-Sort(A, 6, 7)
4
5
6
7
8
96
Quick-Sort(A, 0, 7)
, return
Quick-Sort(A, 5, 7)
4
5
97
Quick-Sort(A, 0, 7)
, return
Quick-Sort(A, 4, 7)
4
98
Quick-Sort(A, 0, 7)
, done!
Quick-Sort(A, 0, 7)
4
99
Quick-Sort Best Case

• Even Partition

Total time ?(nlogn)
100
Quick-Sort Worst Case

• Unbalanced Partition

cn
n
c(n-1)
n-1
c(n-2)
n-2
3c
3

• Happens only if
• input is sortd
• input is reversely sorted

2c
2
Total time ?(n2)
101
Randomized Quick Sort
102
Quick-Sort an Average Case
Quick-Sort an Average Case

• Suppose the split is 1/10 9/10

cn
n
cn
0.1n
0.9n
log10n
0.09n
0.81n
0.01n
0.09n
cn
log10/9n
2
cn
cn
2
Total time ?(nlogn)
103
Quick-Sort Summary

• Time
• Most of the work done in partitioning.
• Average case takes ?(n log(n)) time.
• Worst case takes ?(n2) time
• Space
• Sorts in-place, i.e., does not require additional
  space

104
Summary

• Divide and Conquer
• Merge-Sort
• Most of the work done in Merging
• ?(n log(n)) time
• ?(n) space
• Quick-Sort
• Most of the work done in partitioning
• Average case takes ?(n log(n)) time
• Worst case takes ?(n2) time
• ?(1) space

105
Quiz 6 (Show output)

• public static void main(String args)
• int array 4,18,16,3,5,21,7
• recursiveQuickSort(array, 0, array.length-1)
• System.out.println("The following array should
  be sorted ")
• printList(array)
• System.exit(0)
• public static void recursiveQuickSort(int
  list, int first, int last)
• if(first
• int p partition(list, first, last)
• printList(list)
• recursiveQuickSort(list, first, p-1)
• recursiveQuickSort(list, p1, last)
• Assume that printlist(list) prints the list
  separated by commas.