Outline

1
Outline

• Priority Queues
• Binary Heaps
• Randomized Mergeable Heaps

2
Priority Queues

• A priority queue stores a collection of
  Comparable elements
• Operations
• add(x) Add x to the collection
• findMin() Return the smallest element in the
  collection
• deleteMin() Remove the smallest element in the
  collection
• In the JCF Queue interface, these are
• add(x)/offer(x) add(x)
• peek() findMind()
• remove()/poll() deleteMin()

3
Priority Queues Applications

• We can sort a set of elements using a priority
  queue. How?

• We just need to insert them all into a priority
  queue and then removing them

public static ltTgt void sort (T a) Queue ltTgt
pq new PriorityQueue ltT gt() for (T x a) pq.
add(x) for (int i 0 i lt a. length
i) ai pq. remove ()
4
Priority Queues Applications

• In simulations, priority queues store events
  ordered by time of occurrence

5
Priority Queues Applications

• In simulations, priority queues store events
  ordered by time of occurrence

6
Priority Queues Applications

• In simulations, priority queues store events
  ordered by time of occurrence

7
Priority Queues Applications

• In simulations, priority queues store events
  ordered by time of occurrence

8
Priority Queues Applications

• In simulations, priority queues store events
  ordered by time of occurrence

9
Priority Queues

• Which structure should we use to implement a
  priority queue?

• Priority queues can be implemented as (binary)
  heaps

• A binary heap is a binary tree where each node u
  stores a value u.prio

• Heap property
• u.prio lt u.left.prio and u.prio lt u.right.prio

10
Priority Queues

• A binary heap is a binary tree where each node u
  stores a value u.prio

• Heap property
• u.prio lt u.left.prio and u.prio lt u.right.prio

0.1
0.5
0.3
1.1
0.9
0.4
0.8
5.1
9.2
1.3
2.1
1.9
11
Priority Queues

• A complete heap uses a complete binary tree

• A complete binary tree of height d has up to
  2d1 - 1 nodes
• To store n nodes, we require 2d1 - 1 n

0.1
0.5
0.3
0.4
0.8
1.1
0.9
5.1
9.2
1.3
2.1
1.9

• 2d1 n 1
• d1 log2(n 1)
• d log2(n 1) 1 O(log n)

• A complete heap of size n has height O(log n)

12
Priority Queues

• How can we maps the nodes of a complete binary
  tree to the positions of an array?

• Eytzinger method

• a0 is the root
• the left child of ai is a2i1
• the right child of ai is a2i2
• the parent of ai is a(i-1)/2

13
Eytzinger method

• Eytzinger method

0
1
2
6
5
4
3
10
9
8
7
11
14
13
12
11
5
6
7
8
9
10
13
12
14
2
3
4
0
1
14
Eytzinger method

• Eytzinger method

0
1
2
6
5
4
3
10
9
8
7
11
14
13
12
root
Left child
Right child
11
5
6
7
8
9
10
13
12
14
2
3
4
0
1
a201 a1
a202 a2
i 0
15
Eytzinger method

• Eytzinger method

0
1
2
6
5
4
3
10
9
8
7
11
14
13
12
Left child
Right child
l
f
g
h
i
j
k
n
m
o
c
d
e
a
b
a251 a11
i 5
a202 a12
16
Implicit Binary Heap

• An implicit binary heap represents a complete
  binary heap in an array a using the Eytzinger
  method

0.1
0.3
0.5
1.1
0.9
0.4
0.8
5.1
9.2
1.3
2.1
1.9
1.9
0.8
0.4
2.1
1.3
9.2
5.1
0.3
0.9
1.1
0.1
0.5

• No extra pointers, all the data lives in a

17
Implicit Binary Heap
public class BinaryHeap ltT extends Comparable
ltTgtgt extends AbstractQueue ltTgt protected T
a protected int n protected int left (int i)
return 2i 1 protected int right (int
i) return 2i 2 protected int parent
(int i) return (i -1)/2 ...
18
Finding the minimum

• How to find the minimum value in a heap?

• Finding the minimum value in a heap is easy

• It's stored at the root
• This is a0

public T peek () return a 0

• Runs in O(1) time

19
Inserting elements

• How to insert an element into a heap?

• To insert x into a (implicit binary) heap, we

• add x as a leaf
• while x is smaller than parent(x)
• swap x with its parent

• Runs in O(log n) time

20
Inserting elements

1. add x as a leaf

0.1
0.5
0.3
1.1
0.9
0.4
0.8
5.1
9.2
1.3
2.1
1.9
0.2
0.2
1.9
0.8
0.4
2.1
1.3
9.2
5.1
0.2
0.3
0.9
1.1
0.1
0.5
21
Inserting elements

1. add x as a leaf

• while x is smaller than parent(x)
• swap x with its parent

0.1
0.5
0.3
1.1
0.9
0.4
0.2
5.1
9.2
1.3
2.1
1.9
0.8
0.2
1.9
0.2
0.4
2.1
1.3
9.2
5.1
0.8
0.3
0.9
1.1
0.1
0.5
22
Inserting elements

1. add x as a leaf

• while x is smaller than parent(x)
• swap x with its parent

0.1
0.5
0.2
1.1
0.9
0.4
0.3
5.1
9.2
1.3
2.1
1.9
0.8
0.2
1.9
0.3
0.4
2.1
1.3
9.2
5.1
0.8
0.2
0.9
1.1
0.1
0.5
23
Inserting elements
public boolean offer (T x) if (n 1 gt a.
length ) grow () an x bubbleUp (n
-1) return true protected void bubbleUp (int
i) int p parent (i) while (i gt 0 ai.
compareTo (ap) lt 0) swap (i,p) i p p
parent (i)
24
Removing the minimum

• How can we remove the minimum value from a heap?

• To delete the minimum from a (implicit binary)
  heap, we

• copy xan-1 into a0
• while x is larger than either of its children
• swap x with the smaller of its children

• Runs in O(log n) time

25
Removing the minimum

1. copy xan-1 into a0

0.1
0.5
0.2
1.1
0.9
0.4
0.3
5.1
9.2
1.3
2.1
1.9
0.8
1.9
0.3
0.4
2.1
1.3
9.2
5.1
0.8
0.2
0.9
1.1
0.1
0.5
26
Removing the minimum

1. copy xan-1 into a0

• while x is larger than either of its children
• swap x with the smaller of its children

0.8
0.2
0.5
0.2
0.8
1.1
0.9
0.4
0.3
5.1
9.2
1.3
2.1
1.9
0.8
1.9
0.3
0.4
2.1
1.3
9.2
5.1
0.2
0.2
0.9
1.1
0.8
0.5
0.8
27
Removing the minimum

1. copy xan-1 into a0

• while x is larger than either of its children
• swap x with the smaller of its children

0.2
0.5
0.8
0.3
1.1
0.9
0.4
0.3
0.8
5.1
9.2
1.3
2.1
1.9
0.8
1.9
0.3
0.4
2.1
1.3
9.2
5.1
0.8
0.8
0.9
1.1
0.2
0.5
0.3
0.8
28
Removing the minimum
public T poll () T x a 0 swap (0,
--n) trickleDown (0) return x
29
Removing the minimum
protected void trickleDown (int i) do int j
-1 int r right (i) if (r lt n ar.
compareTo (ai) lt 0) int l left (i) if
(al. compareTo (ar) lt 0) j l else j
r else int l left (i) if (l lt n
al. compareTo (ai) lt 0) j l if (j
gt 0) swap (i, j) i j while (i gt 0)
30
Summary

• Theorem A (implicit) binary heap supports the
  operations add(x) and deleteMin() in O(log n)
  (amortized) time and supports the findMin()
  operation in O(1) time.

31
Building a heap

• How can we build a heap from an unsorted array a?

• How quickly can we make a into an implicit
  binary heap?

• We can insert elements a0,...,an-1 one at a
  time.

public BinaryHeap (T ia) a ia for (int n
1 n lt a. length n) add(an)

• This takes O(1 log(1)) O(1 log(2)) O(1
  log n)
• Runs in O(nlog n) time

Can we do better?
32
Building a heap

• We can do it in O(n) time. How?

• By working bottom up.

• First build n/2 heaps of size 1

• Next build n/4 heaps of size 3

• Next build n/8 heaps of size 7

• build 1 heap of size n

public BinaryHeap (T ia) a ia n a.
length for (int i n/2 i gt 0 i --)
trickleDown (i)
33
Building a heap in linear time (analysis)

• We call trickleDown(i), n/2j times where j is
  the root of a heap of size 2j - 1

• trickleDown(i) then takes O(log(2j - 1)) O(j)
  time

• Total running time is

• (n/4) O(1)

• (n/8) O(2)

• (n/16) O(3)

• 1 O(log n)

• O(n)

34
Heapsort

• The heapsort algorithm for sorting an array a of
  length n

• Make a into a heap in (O(n) time)
• Repeat n times
• Delete the minimum

public T deleteMin () swap (0,
--n) trickleDown (0)

• Each deletion takes O(log n) time
• Runs in O(n log n) time

• Doesn't require any extra space ( does all work
  inside of input array a)

35
Merging two heaps

• We know how to add one element to a heap.

• What can we do if we want to add another heap?

• In other words, how can we merge 2 heaps?

public NodeltTgt merge(NodeltTgt h1, NodeltTgt
h2) while(h2.size()gt0) T u
h2.peek() h2.deleteMin() h1.add(u)

• The cost for deleteMin() and add() is O(log n)

• We perform those operations n times

• Total cost O(n log n)

36
Merging two heaps

• Can we do better?

• Actually we can do it in O(log n). How?

• merge(h1,h2) can be defined recursively

1. If either of h1 or h2 is nil, we just return
   h2 or h1.

1. We take the one with the biggest minimum value
   and merge it either with the right or left child
   of the other.

1. We continue the process recursively.

37
Merging two heaps

• What is the complexity of this operation?

• in an implicit binary heap it would be O(n)

• we need to copy all the elements in a new array

• How can we reduce it?

• Using a heap ordered binary tree structure we
  can reduce the cost to O(log n). Why?

38
Merging two heaps
NodeltTgt merge(NodeltTgt h1, NodeltTgt h2) if
(h1 nil) return h2 if (h2 nil) return
h1 if (compare(h2.x, h1.x) lt 0) return
merge(h2, h1) // now we know h1.x lt h2.x
if (rand.nextBoolean()) h1.left
merge(h1.left, h2) h1.left.parent h1
else h1.right
merge(h1.right, h2) h1.right.parent
h1 return h1
39
Merging two heaps

• Why we need the random function to merge the
  heaps?

• How can we define the add() and remove()
  operations in a heap ordered binary tree ?

public boolean add(T x) NodeltTgt u
newNode() u.x x r merge(u, r)
r.parent nil n return true
public T remove() T x r.x r
merge(r.left, r.right) if (r ! nil)
r.parent nil n-- return x
40
Analysis of merge(h1,h2)

• Lemma The expected length of a random walk in a
  binary tree with n nodes is at most log(n1)

• proof by induction For n0 is true log(01)0

• Suppose it is true for every n0 lt n
• n1 the size of the left sub-tree and n2 n - n1
  -1
• EW 1 log(n11)/2 log(n21)/2
• EW 1 log( (n-1)/2 1)
• 1 log( (n 1)/2 )
• log(n 1)

41
Summary

• Lemma The expected running time of
  merge(h1,h2) is at most O(log n), where n
  h1.size() h2.size()

• Theorem A MeldableHeap implements the
  (priority) Queue interface and supports the
  operations add(x), remove() and merge(h1,h2) in
  O(log n), expected time per operation.

42
Recall Quicksort

• quicksort(S)

• Pick a random element p from S

• Compare every element to p to get 3 sets
• Slt x ? S x lt p
• S x ? S x p
• Sgt x ? S x gt p

• quicksort(Slt)
• quicksort(Sgt)

6
9
8
5
7
p5
7
8
9
5
6
2
1
3
0
4
2
3
4
0
1
S
Sgt
Slt
43
Quicksort
j
i
6
9
0
3
7
5
p5
2
1
8
5
4
j
i
lt
2
1
8
0
4
6
9
5
3
7
5
gt
i
j
i j
2
1
8
0
4
6
9
8
3
7
5
i
j
2
1
3
0
4
6
9
8
3
7
5
6
9
8
3
7
5
2
1
3
0
4
44
Quicksort
6
9
0
3
7
2
1
8
5
4
2
1
3
0
4
6
9
8
7
5
7
9
8
6
5
2
1
3
0
4
5
2
1
4
0
3
7
8
9
6
2
3
4
0
1
7
8
9
6
5
5
2
3
4
0
1
7
8
9
6
7
8
9
5
6
2
3
4
0
1
45
Recall Heapsort

• The heapsort algorithm for sorting an array a of
  length n

• Make a into a heap in (O(n) time)
• Repeat n times
• Delete the minimum

2
1
8
5
4
6
9
0
3
7
6
9
8
3
7
2
1
4
0
5
2
3
4
0
1
7
8
9
5
6
46
Recall Heapsort
6
9
0
3
7
2
1
8
5
4
6
9
8
3
7
2
5
4
0
1
2
5
8
1
4
6
9
3
7
3
5
8
2
4
6
9
7
9
7
6
5
8
3
4
6
7
8
4
5
9
6
9
8
5
7
8
9
6
7
2
3
4
0
1
5
6
47
Recall Heapsort
9
7
6
5
8
3
4
9
6
7
8
4
5
6
9
8
5
7
8
9
6
7
8
7
9
8
9
9
2
3
4
0
1
7
8
9
5
6
48
Mergesort

• The mergesort algorithm for sorting an array a
  of length n

• Divide a in 2 parts a1 and a2
• Recursively, sort a1 and a2 with mergesort
• Merge a1 and a2

2
1
8
5
4
6
9
0
3
7
6
9
0
3
7
2
1
8
5
4
4
5
8
1
2
6
7
9
0
3
7
8
9
5
6
2
3
4
0
1
49
Merging process
i0
i1
i0
i0 lt i1
4
5
8
1
2
i1 lt i0
2
3
4
0
7
8
9
5
6
1
6
7
9
0
3
i1
50
Analysis of Mergesort

• The mergesort algorithm for sorting an array a
  of length n

1. Divide a in 2 parts a1 and a2
2. Recursively, sort a1 and a2 with mergesort
3. Merge a1 and a2

• The merging process ( step 3 ) costs (O(n) time)

• It is performed log n times.

• Total costs (O(n log n) time)

Theorem The Mergesort algorithm can sort an
array of n items in O(n log n) time
51
Sorting Algorithms

• So far, we have seen 3 sorting algorithms

• Quicksort O(n log n) expected time
• Heapsort O(n log n) time
• Mergesort O(n log n) time

• Is there a faster sorting algorithm (maybe O(n)
  time)?

• Answer No and yes

52
Comparision based Sorting Algorithms

• The algorithm Quicksort, Heapsort and Mergesort
  are sort by comparing the elements.

• Every comparison-based sorting algorithm takes
  (n log n) time for some input

• A comparison tree is a full binary tree
• each internal node u is labelled with a pair u.i
  and u.j
• each leaf is labelled with a permutation of 0,
  ... , n 1

53
Comparision based Sorting Algorithms
lt
a0(gt/lt)a1
gt
lt
gt
a1(gt/lt)a2
a0(gt/lt)a2
gt
lt
a0lta1lta2
a0(gt/lt)a2
a1lta0lta2
a1(gt/lt)a2
lt
gt
gt
a0lta2lta1
a2lta0lta1
lt
a1lta2lta0
a2lta1lta0
54
Comparision based Sorting Algorithms

• Lemma Every comparison tree that sorts any
  input of length n has at least n! Leaves

• Theorem Every comparison tree that sorts any
  input of length n has height at least
  (n/2)log2(n/2)

• The height is log2(n!).
• n!O(nn).
• log2(nn) n log2(n).
• The height is O( n log n ).

55
Sorting in O(n) time

• In-class problem
• Design an algorithm that takes an array a of n
  integers in the range 0, ... , k-1 and sorts
  them in O(n k) time

56
Counting sort
int countingSort (int a, int k) int
c new int k for (int i 0 i lt a.
length i ) cai for
(int i 1 i lt k i ) ci
ci -1 int b new int a. length
for (int i a.length -1 i gt 0 i --)
b--cai ai return b
57
Sorting in O(n) time

1. First count the repetition of any possible
   number.
2. Compute the starting position of the number
3. Copy the numbers on the output array

58
Recall Heapsort
9
7
4
2
0
9
1
0
4
6
2
5
9
9
3
9
0
1
7
2
0
0
0
0
0
0
0
0
0
0
1
1
1
1
1
1
2
2
2
1
2
1
3
2
3
4
1
3
5
2
0
1
2
6
7
9
3
4
5
8
2
3
15
3
0
2
1
2
1
1
5
5
8
9
11
12
13
15
20
23
35
81
92
111
121
132
150
155
9
7
4
2
0
9
1
0
4
6
2
5
9
9
3
9
0
1
7
2
0
1
1
0
0
2
3
4
2
2
6
7
7
4
5
9
9
9
9
9
59
Radix sort

• Radix-sort uses the counting sort algorithm to
  sort integers one digit" at a time
• integers have w bits
• digit" has d bits
• uses w/d passes of counting-sort
• Starts by sorting least-significant digits first
• works up to most significant digits
• Correctness depends on fact that counting sort
  is stable
• if ai aj and i lt j then ai appears
  before aj in the output

60
Radix sort

• Theorem The radix-sort algorithm can sort an
  array a of n w-bit integers in O(n 2d) time

• Theorem The radix-sort algorithm can sort an
  array a of n integers in the range 0, ... , nc
  1 in O(cn) time.

61
Summary

• Comparison based sorting algorithms

• Quicksort O(n log n) expected time
• Heapsort O(n log n) time
• Mergesort O(n log n) time

• No comparison based sorting algorithms

• counting-sort can sort an array a of n integers
  in the range 0, ... , k-1 in O(n k) time

• radix-sort can sort an array a of n integers in
  the range 0, ... , nc 1 in O(cn) time.