CSE 326 Data Structures Part 6: Priority Queues, AKA Heaps

1
CSE 326 Data StructuresPart 6Priority Queues,
AKA Heaps

• Henry Kautz
• Autumn 2002

2
Not Quite Queues

• Consider applications
• ordering CPU jobs
• searching for the exit in a maze
• emergency room admission processing
• Problems?
• short jobs should go first
• most promising nodes should be searched first
• most urgent cases should go first

3
Priority Queue ADT

• Priority Queue operations
• create
• destroy
• insert
• deleteMin
• is_empty
• Priority Queue property for two elements in the
  queue, x and y, if x has a lower priority value
  than y, x will be deleted before y

F(7) E(5) D(100) A(4) B(6)
deleteMin
insert
G(9)
C(3)
4
Applications of the Priority Q

• Hold jobs for a printer in order of length
• Store packets on network routers in order of
  urgency
• Simulate events
• Anything greedy

5
Discrete Event Simulation

• An event is a pair (x,t) where x describes the
  event and t is time it should occur
• A discrete event simulator (DES) maintains a set
  S of events which it intends to simulate in time
  order
• repeat
• Find and remove (x0,t0) from S such that t0 is
  minimum
• Do whatever x0 says to do, in the process new
  events (x2,t2)(xk,tk) may be generated
• Insert the new events into S

6
Emergency Room Simulation

• Patient arrive at time t with injury of
  criticality C
• If no patients waiting and a free doctor, assign
  them to doctor and create a future departure
  event else put patient in the Criticality
  priority queue
• Patient departs at time t
• If someone in Criticality queue, pull out most
  critical and assign to doctor create a future
  departure event

arrive(t,c)
patient generator
time queue
criticality (triage) queue
depart(t)
assignpatient to doctor
arrive(t,c)
depart(t)
7
Naïve Priority Queue Data Structures

• Unsorted list
• insert
• deleteMin
• Sorted list
• insert
• deleteMin

8
BST Tree Priority Queue Data Structure

• Regular BST
• insert
• deleteMin
• AVL Tree
• insert
• deleteMin

8
11
5
12
10
6
2
13
4
14
7
9
Can we do better?
9
Binary Heap Priority Q Data Structure

• Heap-order property
• parents key is less than childrens keys
• result minimum is always at the top
• Structure property
• complete tree with fringe nodes packed to the
  left
• result depth is always O(log n) next open
  location always known

How do we find the minimum?
10
Nifty Storage Trick

• Calculations
• child
• parent
• root
• next free

1
2
3
4
7
5
6
8
9
12
10
11
1
2
3
4
5
6
7
8
9
10
11
12
0
2
4
5
7
6
10
8
11
9
12
14
20
12
11
Nifty Storage Trick

• Calculations
• child left 2node
• right2node1
• parent floor(node/2)
• root 1
• next free length1

1
2
3
4
7
5
6
8
9
12
10
11
1
2
3
4
5
6
7
8
9
10
11
12
0
2
4
5
7
6
10
8
11
9
12
14
20
12
12
DeleteMin
pqueue.deleteMin()
2
20
5
4
8
10
6
7
20
14
12
9
11
13
Percolate Down
20
4
5
4
5
20
8
10
6
7
8
10
6
7
14
12
9
11
14
12
9
11
4
4
5
6
5
6
8
10
20
7
8
10
12
7
14
12
9
11
14
20
9
11
14
DeleteMin Code

• Comparable deleteMin()
• x A1
• A1Asize--
• percolateDown(1)
• return x

percolateDown(int hole) tmpAhole while
(2hole lt size) left 2hole right
left 1 if (right lt size
Aright lt Aleft) target right
else target left if (Atarget lt
tmp) Ahole Atarget hole
target else break Ahole
tmp
Trick to avoid repeatedly copying the value at
A1
Move down
runtime
15
Insert
pqueue.insert(3)
2
5
4
8
10
6
7
20
14
12
9
11
3
16
Percolate Up
2
2
5
4
5
4
8
10
6
7
8
3
6
7
20
14
12
9
11
3
20
14
12
9
11
10
2
3
4
8
5
6
7
20
14
12
9
11
10
17
Insert Code

• void insert(Comparable x)
• // Efficiency hack we wont actually put x
• // into the heap until weve located the
  position
• // it goes in. This avoids having to copy it
• // repeatedly during the percolate up.
• int hole size
• // Percolate up
• for( holegt1 x lt Ahole/2 hole hole/2)
• Ahole Ahole/2
• Ahole x

runtime
18
Performance of Binary Heap
Binary heap worst case Binary heap avg case AVL tree worst case BST tree avg case
Insert O(log n) O(1) percolates 1.6 levels O(log n) O(log n)
Delete Min O(log n) O(log n) O(log n) O(log n)

• In practice binary heaps much simpler to code,
  lower constant factor overhead

19
Changing Priorities

• In many applications the priority of an object in
  a priority queue may change over time
• if a job has been sitting in the printer queue
  for a long time increase its priority
• unix renice
• Must have some (separate) way of find the
  position in the queue of the object to change
  (e.g. a hash table)

20
Other Priority Queue Operations

• decreaseKey
• Given the position of an object in the queue,
  increase its priority (lower its key). Fix heap
  property by
• increaseKey
• given the position of an an object in the queue,
  decrease its priority (increase its key). Fix
  heap property by
• remove
• given the position of an an object in the queue,
  remove it. Do increaseKey to infinity then

21
BuildHeap

• Task Given a set of n keys, build a heap all at
  once
• Approach 1 Repeatedly perform Insert(key)
• Complexity

22
BuildHeapFloyds Method
5
11
3
10
6
9
4
8
1
7
2
12
pretend its a heap and fix the heap-order
property!
12
buildHeap() for (isize/2 igt0 i--)
percolateDown(i)
11
5
9
6
10
3
2
7
1
8
4
23
Build(this)Heap
12
12
11
5
11
5
9
2
10
3
9
2
1
3
6
7
1
8
4
6
7
10
8
4
12
12
2
5
2
1
9
6
1
3
9
6
5
3
11
7
10
8
4
11
7
10
8
4
24
Finally
1
2
3
9
6
5
4
11
7
10
8
12
25
Complexity of Build Heap

• Note size of a perfect binary tree doubles (1)
  with each additional layer
• At most n/4 percolate down 1 levelat most n/8
  percolate down 2 levelsat most n/16 percolate
  down 3 levels

O(n)
26
Heap Sort

• Input unordered array A1..N
• Build a max heap (largest element is A1)
• For i 1 to N-1 AN-i1 Delete_Max()

7
50
22
15
4
40
20
10
35
25
50
40
20
25
35
15
10
22
4
7
40
35
20
25
7
15
10
22
4
50
35
25
20
22
7
15
10
4
40
50
27
Properties of Heap Sort

• Worst case time complexity O(n log n)
• Build_heap O(n)
• n Delete_Maxs for O(n log n)
• In-place sort only constant storage beyond the
  array is needed

28
Thinking about Heaps

• Observations
• finding a child/parent index is a multiply/divide
  by two
• operations jump widely through the heap
• each operation looks at only two new nodes
• inserts are at least as common as deleteMins
• Realities
• division and multiplication by powers of two are
  fast
• looking at one new piece of data terrible in a
  cache line
• with huge data sets, disk accesses dominate

29
Solution d-Heaps
1

• Each node has d children
• Still representable by array
• Good choices for d
• optimize performance based on of
  inserts/removes
• choose a power of two for efficiency
• fit one set of children in a cache line
• fit one set of children on a memory page/disk
  block

2
3
7
9
6
5
4
8
10
12
11
4
3
7
2
8
5
12
11
10
6
9
1
12
30
Coming Up

• Mergeable heaps
• Leftist heaps
• Skew heaps
• Binomial queues
• Read Weiss Ch. 6
• Midterm results

31
New Operation Merge

• Merge(H1,H2) Merge two heaps H1 and H2 of size
  O(N).
• E.g. Combine queues from two different sources to
  run on one CPU.
• Can do O(N) Insert operations O(N log N) time
• Better Copy H2 at the end of H1 (assuming array
  implementation) and use Floyds Method for
  BuildHeap.
• Running Time O(N)
• Can we do even better? (i.e. Merge in O(log N)
  time?)

32
Binomial Queues

• Binomial queues support all three priority queue
  operations Merge, Insert and DeleteMin in O(log
  N) time
• Idea Maintain a collection of heap-ordered trees
• Forest of binomial trees
• Recursive Definition of Binomial Tree (based on
  height k)
• Only one binomial tree for a given height
• Binomial tree of height 0 single root node
• Binomial tree of height k Bk Attach Bk-1 to
  root of another Bk-1

33
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
34
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
35
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
36
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
37
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
38
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
39
Building a Binomial Tree

• To construct a binomial tree Bk of height k
• Take the binomial tree Bk-1 of height k-1
• Place another copy of Bk-1 one level below the
  first
• Attach the root nodes
• Binomial tree of height k has exactly 2k nodes
  (by induction)

B0 B1 B2 B3
40
Why Binomial?

• Why are these trees called binomial?
• Hint how many nodes at depth d?

B0 B1 B2 B3
41
Why Binomial?

• Why are these trees called binomial?
• Hint how many nodes at depth d?
• Number of nodes at different depths d for Bk
• 1, 1 1, 1 2 1, 1 3 3 1,
• Binomial coefficients of (a b)k k!/((k-d)!d!)

B0 B1 B2 B3
42
Definition of Binomial Queues
Binomial Queue forest of heap-ordered
binomial trees
B0 B2 B0 B1 B3
1
-1
21
5
3
3
7
2
1
9
6
11
5
8
7
Binomial queue H1 5 elements 101 base 2 ? B2 B0
Binomial queue H2 11 elements 1011 base 2 ? B3
B1 B0
6
43
Binomial Queue Properties

• Suppose you are given a binomial queue of N nodes
• There is a unique set of binomial trees for N
  nodes
• What is the maximum number of trees that can be
  in an N-node queue?
• 1 node ? 1 tree B0 2 nodes ? 1 tree B1 3 nodes
  ? 2 trees B0 and B1 7 nodes ? 3 trees B0, B1 and
  B2
• Trees B0, B1, , Bk can store up to 20 21
  2k 2k1 1 nodes N.
• Maximum is when all trees are used. So, solve
  for (k1).
• Number of trees is ? log(N1) O(log N)

44
Binomial Queues Merge

• Main Idea Merge two binomial queues by merging
  individual binomial trees
• Since Bk1 is just two Bks attached together,
  merging trees is easy
• Steps for creating new queue by merging
• Start with Bk for smallest k in either queue.
• If only one Bk, add Bk to new queue and go to
  next k.
• Merge two Bks to get new Bk1 by making larger
  root the child of smaller root. Go to step 2 with
  k k 1.

45
Example Binomial Queue Merge
H1 H2
1
-1
5
3
21
3
9
7
2
1
6
11
5
7
8
6
46
Example Binomial Queue Merge
H1 H2
1
-1
5
3
3
9
7
2
1
6
21
11
5
7
8
6
47
Example Binomial Queue Merge
H1 H2
1
-1
5
3
9
7
2
1
6
3
11
5
7
8
21
6
48
Example Binomial Queue Merge
H1 H2
1
-1
3
5
7
2
1
3
11
5
9
6
8
21
6
7
49
Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
50
Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
51
Binomial Queues Merge and Insert

• What is the run time for Merge of two O(N)
  queues?
• How would you insert a new item into the queue?

52
Binomial Queues Merge and Insert

• What is the run time for Merge of two O(N)
  queues?
• O(number of trees) O(log N)
• How would you insert a new item into the queue?
• Create a single node queue B0 with new item and
  merge with existing queue
• Again, O(log N) time
• Example Insert 1, 2, 3, ,7 into an empty
  binomial queue

53
Insert 1,2,,7
1
54
Insert 1,2,,7
1
2
55
Insert 1,2,,7
3
1
2
56
Insert 1,2,,7
3
1
2
4
57
Insert 1,2,,7
1
2
3
4
58
Insert 1,2,,7
1
5
2
3
4
59
Insert 1,2,,7
1
5
2
3
6
4
60
Insert 1,2,,7
1
5
7
2
3
6
4
61
Binomial Queues DeleteMin

• Steps
• Find tree Bk with the smallest root
• Remove Bk from the queue
• Delete root of Bk (return this value) You now
  have a new queue made up of the forest B0, B1, ,
  Bk-1
• Merge this queue with remainder of the original
  (from step 2)
• Run time analysis Step 1 is O(log N), step 2 and
  3 are O(1), and step 4 is O(log N). Total time
  O(log N)
• Example Insert 1, 2, , 7 into empty queue and
  DeleteMin

62
Insert 1,2,,7
1
5
7
2
3
6
4
63
DeleteMin
5
7
2
3
6
4
64
Merge
5
2
3
6
4
7
65
Merge
5
2
3
6
4
7
66
Merge
5
2
6
3
7
4
67
Merge
5
2
6
3
7
DONE!
4
68
Implementation of Binomial Queues

• Need to be able to scan through all trees, and
  given two binomial queues find trees that are
  same size
• Use array of pointers to root nodes, sorted by
  size
• Since is only of length log(N), dont have to
  worry about cost of copying this array
• At each node, keep track of the size of the (sub)
  tree rooted at that node
• Want to merge by just setting pointers
• Need pointer-based implementation of heaps
• DeleteMin requires fast access to all subtrees of
  root
• Use First-Child/Next-Sibling representation of
  trees

69
Efficient BuildHeap for Binomial Queues

• Insert one at a time - O(n log n)
• Better algorithm
• Start with each element as a singleton tree
• Merge trees of size 1
• Merge trees of size 2
• Merge trees of size 4
• Complexity

70
Other Mergeable Priority Queues Leftist and
Skew Heaps

• Leftist Heaps Binary heap-ordered trees with
  left subtrees always longer than right subtrees
• Main idea Recursively work on right path for
  Merge/Insert/DeleteMin
• Right path is always short ? has O(log N) nodes
• Merge, Insert, DeleteMin all have O(log N)
  running time (see text)
• Skew Heaps Self-adjusting version of leftist
  heaps (a la splay trees)
• Do not actually keep track of path lengths
• Adjust tree by swapping children during each
  merge
• O(log N) amortized time per operation for a
  sequence of M operations
• See Weiss for details

71
Coming Up

• For Wednesday Read Weiss on Leftist Heaps and
  Skew Heaps
• How do they work?
• How does their complexity compare to Binomial
  Queues?
• Which seems easiest to implement?
• Which is likely to have lowest overhead?

72
Leftist Heaps

• An alternative heap structure that also enables
  fast merges
• Based on binary trees rather than k-ary trees

73
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
?
Now, just percolate down!
1
2
4
9
11
5
14
12
13
10
6
10
74
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
1
Now, just percolate down!
?
2
4
9
11
5
14
12
13
10
6
10
75
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
1
Now, just percolate down!
4
2
?
9
11
5
14
12
13
10
6
10
76
Idea Hang a New Tree
2
1


11
5
4
9
12
13
10
6
10
14
1
Note we just gave up the nice structural
property on binary heaps!
Now, just percolate down!
4
2
9
11
5
14
12
13
10
6
10
77
Problem?
1
2


4
12
15

• Need some other kind of balance condition

15
18
?
1
2
4
12
15
15
18
78
Leftist Heaps

• Idea make it so that all the work you have to do
  in maintaining a heap is in one small part
• Leftist heap
• almost all nodes are on the left
• all the merging work is on the right

79
Random DefinitionNull Path Length
the null path length (npl) of a node is the
number of nodes between it and a null in the tree

• npl(null) -1
• npl(leaf) 0
• npl(single-child node) 0

2
1
1
0
0
0
1
another way of looking at it npl is the height
of complete subtree rooted at this node
0
0
0
80
Leftist Heap Properties

• Heap-order property
• parents priority value is ? to childrens
  priority values
• result minimum element is at the root
• Leftist property
• null path length of left subtree is ? npl of
  right subtree
• result tree is at least as heavy on the left
  as the right

Are leftist trees complete? Balanced?
81
Leftist tree examples
NOT leftist
leftist
leftist
2
2
0
1
1
1
1
0
0
0
1
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
every subtree of a leftist tree is leftist,
comrade!
0
82
Right Path in a Leftist Tree is Short
2

• If the right path has length
• at least r, the tree has at
• least 2r - 1 nodes
• Proof by induction
• Basis r 1. Tree has at least one node 21 - 1
  1
• Inductive step assume true for r lt r. The
  right subtree has a right path of at least r - 1
  nodes, so it has at least 2r - 1 - 1 nodes. The
  left subtree must also have a right path of at
  least r - 1 (otherwise, there is a null path of r
  - 3, less than the right subtree). Again, the
  left has 2r - 1 - 1 nodes. All told then, there
  are at least
• 2r - 1 - 1 2r - 1 - 1 1 2r - 1
• So, a leftist tree with at least n nodes has a
  right path of at most log n nodes

1
1
0
0
0
1
0
0
0
83
Merging Two Leftist Heaps

• merge(T1,T2) returns one leftist heap containing
  all elements of the two (distinct) leftist heaps
  T1 and T2

merge
T1
a
a
recursive merge
L1
R1
L1
R1
a lt b
T2
b
b
L2
R2
L2
R2
84
Merge Continued
a
a
npl(R) gt npl(L1)
L1
R
R
L1
R Merge(R1, T2)
constant work at each merge recursively traverse
RIGHT right path of each tree total work O(log
n)
85
Operations on Leftist Heaps

• merge with two trees of total size n O(log n)
• insert with heap size n O(log n)
• pretend node is a size 1 leftist heap
• insert by merging original heap with one node
  heap
• deleteMin with heap size n O(log n)
• remove and return root
• merge left and right subtrees

merge
merge
86
Example
merge
?
1
3
5
merge
0
0
0
7
1
12
10
?
5
5
0
1
merge
14
0
0
0
3
12
10
10
0
12
0
0
0
8
7
0
8
8
0
14
0
8
0
12
87
Sewing Up the Example
?
?
2
3
3
3
0
0
0
?
7
1
1
7
7
5
5
5
0
0
0
0
0
0
0
0
14
14
8
14
10
0
10
8
10
8
0
0
12
0
12
12
Done?
88
Finally
2
2
3
3
0
0
1
1
7
7
5
5
0
0
0
0
0
0
8
14
14
8
10
10
0
0
12
12
89
Skew Heaps

• Problems with leftist heaps
• extra storage for npl
• two pass merge (with stack!)
• extra complexity/logic to maintain and check npl
• Solution skew heaps
• blind adjusting version of leftist heaps
• amortized time for merge, insert, and deleteMin
  is O(log n)
• worst case time for all three is O(n)
• merge always switches children when fixing right
  path
• iterative method has only one pass

What do skew heaps remind us of?
90
Merging Two Skew Heaps
merge
T1
a
a
merge
L1
R1
L1
R1
a lt b
T2
b
b
Notice the old switcheroo!
L2
R2
L2
R2
91
Example
merge
3
3
5
merge
7
7
5
12
10
5
merge
14
14
10
3
12
12
10
8
8
7
8
3
14
7
5
14
10
8
12
92
Skew Heap Code

• void merge(heap1, heap2)
• case
• heap1 NULL return heap2
• heap2 NULL return heap1
• heap1.findMin() lt heap2.findMin()
• temp heap1.right
• heap1.right heap1.left
• heap1.left merge(heap2, temp)
• return heap1
• otherwise
• return merge(heap2, heap1)

93
Comparing Heaps

• Binary Heaps
• d-Heaps
• Binomial Queues

• Leftist Heaps
• Skew Heaps

94
Summary of Heap ADT Analysis

• Consider a heap of N nodes
• Space needed O(N)
• Actually, O(MaxSize) where MaxSize is the size of
  the array
• Pointer-based implementation pointers for
  children and parent
• Total space 3N 1 (3 pointers per node 1 for
  size)
• FindMin O(1) time DeleteMin and Insert O(log
  N) time
• BuildHeap from N inputs What is the run time?
• N Insert operations O(N log N)
• O(N) Treat input array as a heap and fix it
  using percolate down
• Thanks, Floyd!
• Mergable Heaps Binomial Queues, Leftist Heaps,
  Skew Heaps