Title: CSE 326 Data Structures Part 6: Priority Queues, AKA Heaps
1CSE 326 Data StructuresPart 6Priority Queues,
AKA Heaps
2Not Quite Queues
- Consider applications
- ordering CPU jobs
- searching for the exit in a maze
- emergency room admission processing
- Problems?
- short jobs should go first
- most promising nodes should be searched first
- most urgent cases should go first
3Priority Queue ADT
- Priority Queue operations
- create
- destroy
- insert
- deleteMin
- is_empty
- Priority Queue property for two elements in the
queue, x and y, if x has a lower priority value
than y, x will be deleted before y
F(7) E(5) D(100) A(4) B(6)
deleteMin
insert
G(9)
C(3)
4Applications of the Priority Q
- Hold jobs for a printer in order of length
- Store packets on network routers in order of
urgency - Simulate events
- Anything greedy
5Discrete Event Simulation
- An event is a pair (x,t) where x describes the
event and t is time it should occur - A discrete event simulator (DES) maintains a set
S of events which it intends to simulate in time
order - repeat
- Find and remove (x0,t0) from S such that t0 is
minimum - Do whatever x0 says to do, in the process new
events (x2,t2)(xk,tk) may be generated - Insert the new events into S
6Emergency Room Simulation
- Patient arrive at time t with injury of
criticality C - If no patients waiting and a free doctor, assign
them to doctor and create a future departure
event else put patient in the Criticality
priority queue - Patient departs at time t
- If someone in Criticality queue, pull out most
critical and assign to doctor create a future
departure event
arrive(t,c)
patient generator
time queue
criticality (triage) queue
depart(t)
assignpatient to doctor
arrive(t,c)
depart(t)
7Naïve Priority Queue Data Structures
- Unsorted list
- insert
- deleteMin
- Sorted list
- insert
- deleteMin
8BST Tree Priority Queue Data Structure
- Regular BST
- insert
- deleteMin
- AVL Tree
- insert
- deleteMin
8
11
5
12
10
6
2
13
4
14
7
9
Can we do better?
9Binary Heap Priority Q Data Structure
- Heap-order property
- parents key is less than childrens keys
- result minimum is always at the top
- Structure property
- complete tree with fringe nodes packed to the
left - result depth is always O(log n) next open
location always known
How do we find the minimum?
10Nifty Storage Trick
- Calculations
- child
- parent
- root
- next free
1
2
3
4
7
5
6
8
9
12
10
11
1
2
3
4
5
6
7
8
9
10
11
12
0
2
4
5
7
6
10
8
11
9
12
14
20
12
11Nifty Storage Trick
- Calculations
- child left 2node
- right2node1
- parent floor(node/2)
- root 1
- next free length1
1
2
3
4
7
5
6
8
9
12
10
11
1
2
3
4
5
6
7
8
9
10
11
12
0
2
4
5
7
6
10
8
11
9
12
14
20
12
12DeleteMin
pqueue.deleteMin()
2
20
5
4
8
10
6
7
20
14
12
9
11
13Percolate Down
20
4
5
4
5
20
8
10
6
7
8
10
6
7
14
12
9
11
14
12
9
11
4
4
5
6
5
6
8
10
20
7
8
10
12
7
14
12
9
11
14
20
9
11
14DeleteMin Code
- Comparable deleteMin()
- x A1
- A1Asize--
- percolateDown(1)
- return x
percolateDown(int hole) tmpAhole while
(2hole lt size) left 2hole right
left 1 if (right lt size
Aright lt Aleft) target right
else target left if (Atarget lt
tmp) Ahole Atarget hole
target else break Ahole
tmp
Trick to avoid repeatedly copying the value at
A1
Move down
runtime
15Insert
pqueue.insert(3)
2
5
4
8
10
6
7
20
14
12
9
11
3
16Percolate Up
2
2
5
4
5
4
8
10
6
7
8
3
6
7
20
14
12
9
11
3
20
14
12
9
11
10
2
3
4
8
5
6
7
20
14
12
9
11
10
17Insert Code
- void insert(Comparable x)
- // Efficiency hack we wont actually put x
- // into the heap until weve located the
position - // it goes in. This avoids having to copy it
- // repeatedly during the percolate up.
- int hole size
- // Percolate up
- for( holegt1 x lt Ahole/2 hole hole/2)
- Ahole Ahole/2
- Ahole x
runtime
18Performance of Binary Heap
Binary heap worst case Binary heap avg case AVL tree worst case BST tree avg case
Insert O(log n) O(1) percolates 1.6 levels O(log n) O(log n)
Delete Min O(log n) O(log n) O(log n) O(log n)
- In practice binary heaps much simpler to code,
lower constant factor overhead
19Changing Priorities
- In many applications the priority of an object in
a priority queue may change over time - if a job has been sitting in the printer queue
for a long time increase its priority - unix renice
- Must have some (separate) way of find the
position in the queue of the object to change
(e.g. a hash table)
20Other Priority Queue Operations
- decreaseKey
- Given the position of an object in the queue,
increase its priority (lower its key). Fix heap
property by - increaseKey
- given the position of an an object in the queue,
decrease its priority (increase its key). Fix
heap property by - remove
- given the position of an an object in the queue,
remove it. Do increaseKey to infinity then
21BuildHeap
- Task Given a set of n keys, build a heap all at
once - Approach 1 Repeatedly perform Insert(key)
- Complexity
22BuildHeapFloyds Method
5
11
3
10
6
9
4
8
1
7
2
12
pretend its a heap and fix the heap-order
property!
12
buildHeap() for (isize/2 igt0 i--)
percolateDown(i)
11
5
9
6
10
3
2
7
1
8
4
23Build(this)Heap
12
12
11
5
11
5
9
2
10
3
9
2
1
3
6
7
1
8
4
6
7
10
8
4
12
12
2
5
2
1
9
6
1
3
9
6
5
3
11
7
10
8
4
11
7
10
8
4
24Finally
1
2
3
9
6
5
4
11
7
10
8
12
25Complexity of Build Heap
- Note size of a perfect binary tree doubles (1)
with each additional layer - At most n/4 percolate down 1 levelat most n/8
percolate down 2 levelsat most n/16 percolate
down 3 levels
O(n)
26Heap Sort
- Input unordered array A1..N
- Build a max heap (largest element is A1)
- For i 1 to N-1 AN-i1 Delete_Max()
7
50
22
15
4
40
20
10
35
25
50
40
20
25
35
15
10
22
4
7
40
35
20
25
7
15
10
22
4
50
35
25
20
22
7
15
10
4
40
50
27Properties of Heap Sort
- Worst case time complexity O(n log n)
- Build_heap O(n)
- n Delete_Maxs for O(n log n)
- In-place sort only constant storage beyond the
array is needed
28Thinking about Heaps
- Observations
- finding a child/parent index is a multiply/divide
by two - operations jump widely through the heap
- each operation looks at only two new nodes
- inserts are at least as common as deleteMins
- Realities
- division and multiplication by powers of two are
fast - looking at one new piece of data terrible in a
cache line - with huge data sets, disk accesses dominate
29Solution d-Heaps
1
- Each node has d children
- Still representable by array
- Good choices for d
- optimize performance based on of
inserts/removes - choose a power of two for efficiency
- fit one set of children in a cache line
- fit one set of children on a memory page/disk
block
2
3
7
9
6
5
4
8
10
12
11
4
3
7
2
8
5
12
11
10
6
9
1
12
30Coming Up
- Mergeable heaps
- Leftist heaps
- Skew heaps
- Binomial queues
- Read Weiss Ch. 6
- Midterm results
31New Operation Merge
- Merge(H1,H2) Merge two heaps H1 and H2 of size
O(N). - E.g. Combine queues from two different sources to
run on one CPU. - Can do O(N) Insert operations O(N log N) time
- Better Copy H2 at the end of H1 (assuming array
implementation) and use Floyds Method for
BuildHeap. - Running Time O(N)
- Can we do even better? (i.e. Merge in O(log N)
time?)
32Binomial Queues
- Binomial queues support all three priority queue
operations Merge, Insert and DeleteMin in O(log
N) time - Idea Maintain a collection of heap-ordered trees
- Forest of binomial trees
- Recursive Definition of Binomial Tree (based on
height k) - Only one binomial tree for a given height
- Binomial tree of height 0 single root node
- Binomial tree of height k Bk Attach Bk-1 to
root of another Bk-1
33Building a Binomial Tree
- To construct a binomial tree Bk of height k
- Take the binomial tree Bk-1 of height k-1
- Place another copy of Bk-1 one level below the
first - Attach the root nodes
- Binomial tree of height k has exactly 2k nodes
(by induction)
B0 B1 B2 B3
34Building a Binomial Tree
- To construct a binomial tree Bk of height k
- Take the binomial tree Bk-1 of height k-1
- Place another copy of Bk-1 one level below the
first - Attach the root nodes
- Binomial tree of height k has exactly 2k nodes
(by induction)
B0 B1 B2 B3
35Building a Binomial Tree
- To construct a binomial tree Bk of height k
- Take the binomial tree Bk-1 of height k-1
- Place another copy of Bk-1 one level below the
first - Attach the root nodes
- Binomial tree of height k has exactly 2k nodes
(by induction)
B0 B1 B2 B3
36Building a Binomial Tree
- To construct a binomial tree Bk of height k
- Take the binomial tree Bk-1 of height k-1
- Place another copy of Bk-1 one level below the
first - Attach the root nodes
- Binomial tree of height k has exactly 2k nodes
(by induction)
B0 B1 B2 B3
37Building a Binomial Tree
- To construct a binomial tree Bk of height k
- Take the binomial tree Bk-1 of height k-1
- Place another copy of Bk-1 one level below the
first - Attach the root nodes
- Binomial tree of height k has exactly 2k nodes
(by induction)
B0 B1 B2 B3
38Building a Binomial Tree
- To construct a binomial tree Bk of height k
- Take the binomial tree Bk-1 of height k-1
- Place another copy of Bk-1 one level below the
first - Attach the root nodes
- Binomial tree of height k has exactly 2k nodes
(by induction)
B0 B1 B2 B3
39Building a Binomial Tree
- To construct a binomial tree Bk of height k
- Take the binomial tree Bk-1 of height k-1
- Place another copy of Bk-1 one level below the
first - Attach the root nodes
- Binomial tree of height k has exactly 2k nodes
(by induction)
B0 B1 B2 B3
40Why Binomial?
- Why are these trees called binomial?
- Hint how many nodes at depth d?
B0 B1 B2 B3
41Why Binomial?
- Why are these trees called binomial?
- Hint how many nodes at depth d?
- Number of nodes at different depths d for Bk
- 1, 1 1, 1 2 1, 1 3 3 1,
- Binomial coefficients of (a b)k k!/((k-d)!d!)
B0 B1 B2 B3
42Definition of Binomial Queues
Binomial Queue forest of heap-ordered
binomial trees
B0 B2 B0 B1 B3
1
-1
21
5
3
3
7
2
1
9
6
11
5
8
7
Binomial queue H1 5 elements 101 base 2 ? B2 B0
Binomial queue H2 11 elements 1011 base 2 ? B3
B1 B0
6
43Binomial Queue Properties
- Suppose you are given a binomial queue of N nodes
- There is a unique set of binomial trees for N
nodes - What is the maximum number of trees that can be
in an N-node queue? - 1 node ? 1 tree B0 2 nodes ? 1 tree B1 3 nodes
? 2 trees B0 and B1 7 nodes ? 3 trees B0, B1 and
B2 - Trees B0, B1, , Bk can store up to 20 21
2k 2k1 1 nodes N. - Maximum is when all trees are used. So, solve
for (k1). - Number of trees is ? log(N1) O(log N)
44Binomial Queues Merge
- Main Idea Merge two binomial queues by merging
individual binomial trees - Since Bk1 is just two Bks attached together,
merging trees is easy - Steps for creating new queue by merging
- Start with Bk for smallest k in either queue.
- If only one Bk, add Bk to new queue and go to
next k. - Merge two Bks to get new Bk1 by making larger
root the child of smaller root. Go to step 2 with
k k 1.
45Example Binomial Queue Merge
H1 H2
1
-1
5
3
21
3
9
7
2
1
6
11
5
7
8
6
46Example Binomial Queue Merge
H1 H2
1
-1
5
3
3
9
7
2
1
6
21
11
5
7
8
6
47Example Binomial Queue Merge
H1 H2
1
-1
5
3
9
7
2
1
6
3
11
5
7
8
21
6
48Example Binomial Queue Merge
H1 H2
1
-1
3
5
7
2
1
3
11
5
9
6
8
21
6
7
49Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
50Example Binomial Queue Merge
H1 H2
-1
1
3
2
1
5
7
3
11
5
8
9
6
6
21
7
51Binomial Queues Merge and Insert
- What is the run time for Merge of two O(N)
queues? - How would you insert a new item into the queue?
52Binomial Queues Merge and Insert
- What is the run time for Merge of two O(N)
queues? - O(number of trees) O(log N)
- How would you insert a new item into the queue?
- Create a single node queue B0 with new item and
merge with existing queue - Again, O(log N) time
- Example Insert 1, 2, 3, ,7 into an empty
binomial queue
53Insert 1,2,,7
1
54Insert 1,2,,7
1
2
55Insert 1,2,,7
3
1
2
56Insert 1,2,,7
3
1
2
4
57Insert 1,2,,7
1
2
3
4
58Insert 1,2,,7
1
5
2
3
4
59Insert 1,2,,7
1
5
2
3
6
4
60Insert 1,2,,7
1
5
7
2
3
6
4
61Binomial Queues DeleteMin
- Steps
- Find tree Bk with the smallest root
- Remove Bk from the queue
- Delete root of Bk (return this value) You now
have a new queue made up of the forest B0, B1, ,
Bk-1 - Merge this queue with remainder of the original
(from step 2) - Run time analysis Step 1 is O(log N), step 2 and
3 are O(1), and step 4 is O(log N). Total time
O(log N) - Example Insert 1, 2, , 7 into empty queue and
DeleteMin
62Insert 1,2,,7
1
5
7
2
3
6
4
63DeleteMin
5
7
2
3
6
4
64Merge
5
2
3
6
4
7
65Merge
5
2
3
6
4
7
66Merge
5
2
6
3
7
4
67Merge
5
2
6
3
7
DONE!
4
68Implementation of Binomial Queues
- Need to be able to scan through all trees, and
given two binomial queues find trees that are
same size - Use array of pointers to root nodes, sorted by
size - Since is only of length log(N), dont have to
worry about cost of copying this array - At each node, keep track of the size of the (sub)
tree rooted at that node - Want to merge by just setting pointers
- Need pointer-based implementation of heaps
- DeleteMin requires fast access to all subtrees of
root - Use First-Child/Next-Sibling representation of
trees
69Efficient BuildHeap for Binomial Queues
- Insert one at a time - O(n log n)
- Better algorithm
- Start with each element as a singleton tree
- Merge trees of size 1
- Merge trees of size 2
- Merge trees of size 4
- Complexity
70Other Mergeable Priority Queues Leftist and
Skew Heaps
- Leftist Heaps Binary heap-ordered trees with
left subtrees always longer than right subtrees - Main idea Recursively work on right path for
Merge/Insert/DeleteMin - Right path is always short ? has O(log N) nodes
- Merge, Insert, DeleteMin all have O(log N)
running time (see text) - Skew Heaps Self-adjusting version of leftist
heaps (a la splay trees) - Do not actually keep track of path lengths
- Adjust tree by swapping children during each
merge - O(log N) amortized time per operation for a
sequence of M operations - See Weiss for details
71Coming Up
- For Wednesday Read Weiss on Leftist Heaps and
Skew Heaps - How do they work?
- How does their complexity compare to Binomial
Queues? - Which seems easiest to implement?
- Which is likely to have lowest overhead?
72Leftist Heaps
- An alternative heap structure that also enables
fast merges - Based on binary trees rather than k-ary trees
73Idea Hang a New Tree
2
1
11
5
4
9
12
13
10
6
10
14
?
Now, just percolate down!
1
2
4
9
11
5
14
12
13
10
6
10
74Idea Hang a New Tree
2
1
11
5
4
9
12
13
10
6
10
14
1
Now, just percolate down!
?
2
4
9
11
5
14
12
13
10
6
10
75Idea Hang a New Tree
2
1
11
5
4
9
12
13
10
6
10
14
1
Now, just percolate down!
4
2
?
9
11
5
14
12
13
10
6
10
76Idea Hang a New Tree
2
1
11
5
4
9
12
13
10
6
10
14
1
Note we just gave up the nice structural
property on binary heaps!
Now, just percolate down!
4
2
9
11
5
14
12
13
10
6
10
77Problem?
1
2
4
12
15
- Need some other kind of balance condition
15
18
?
1
2
4
12
15
15
18
78Leftist Heaps
- Idea make it so that all the work you have to do
in maintaining a heap is in one small part - Leftist heap
- almost all nodes are on the left
- all the merging work is on the right
79Random DefinitionNull Path Length
the null path length (npl) of a node is the
number of nodes between it and a null in the tree
- npl(null) -1
- npl(leaf) 0
- npl(single-child node) 0
2
1
1
0
0
0
1
another way of looking at it npl is the height
of complete subtree rooted at this node
0
0
0
80Leftist Heap Properties
- Heap-order property
- parents priority value is ? to childrens
priority values - result minimum element is at the root
- Leftist property
- null path length of left subtree is ? npl of
right subtree - result tree is at least as heavy on the left
as the right
Are leftist trees complete? Balanced?
81Leftist tree examples
NOT leftist
leftist
leftist
2
2
0
1
1
1
1
0
0
0
1
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
every subtree of a leftist tree is leftist,
comrade!
0
82Right Path in a Leftist Tree is Short
2
- If the right path has length
- at least r, the tree has at
- least 2r - 1 nodes
- Proof by induction
- Basis r 1. Tree has at least one node 21 - 1
1 - Inductive step assume true for r lt r. The
right subtree has a right path of at least r - 1
nodes, so it has at least 2r - 1 - 1 nodes. The
left subtree must also have a right path of at
least r - 1 (otherwise, there is a null path of r
- 3, less than the right subtree). Again, the
left has 2r - 1 - 1 nodes. All told then, there
are at least - 2r - 1 - 1 2r - 1 - 1 1 2r - 1
- So, a leftist tree with at least n nodes has a
right path of at most log n nodes
1
1
0
0
0
1
0
0
0
83Merging Two Leftist Heaps
- merge(T1,T2) returns one leftist heap containing
all elements of the two (distinct) leftist heaps
T1 and T2
merge
T1
a
a
recursive merge
L1
R1
L1
R1
a lt b
T2
b
b
L2
R2
L2
R2
84Merge Continued
a
a
npl(R) gt npl(L1)
L1
R
R
L1
R Merge(R1, T2)
constant work at each merge recursively traverse
RIGHT right path of each tree total work O(log
n)
85Operations on Leftist Heaps
- merge with two trees of total size n O(log n)
- insert with heap size n O(log n)
- pretend node is a size 1 leftist heap
- insert by merging original heap with one node
heap -
- deleteMin with heap size n O(log n)
- remove and return root
- merge left and right subtrees
merge
merge
86Example
merge
?
1
3
5
merge
0
0
0
7
1
12
10
?
5
5
0
1
merge
14
0
0
0
3
12
10
10
0
12
0
0
0
8
7
0
8
8
0
14
0
8
0
12
87Sewing Up the Example
?
?
2
3
3
3
0
0
0
?
7
1
1
7
7
5
5
5
0
0
0
0
0
0
0
0
14
14
8
14
10
0
10
8
10
8
0
0
12
0
12
12
Done?
88Finally
2
2
3
3
0
0
1
1
7
7
5
5
0
0
0
0
0
0
8
14
14
8
10
10
0
0
12
12
89Skew Heaps
- Problems with leftist heaps
- extra storage for npl
- two pass merge (with stack!)
- extra complexity/logic to maintain and check npl
- Solution skew heaps
- blind adjusting version of leftist heaps
- amortized time for merge, insert, and deleteMin
is O(log n) - worst case time for all three is O(n)
- merge always switches children when fixing right
path - iterative method has only one pass
What do skew heaps remind us of?
90Merging Two Skew Heaps
merge
T1
a
a
merge
L1
R1
L1
R1
a lt b
T2
b
b
Notice the old switcheroo!
L2
R2
L2
R2
91Example
merge
3
3
5
merge
7
7
5
12
10
5
merge
14
14
10
3
12
12
10
8
8
7
8
3
14
7
5
14
10
8
12
92Skew Heap Code
- void merge(heap1, heap2)
- case
- heap1 NULL return heap2
- heap2 NULL return heap1
- heap1.findMin() lt heap2.findMin()
- temp heap1.right
- heap1.right heap1.left
- heap1.left merge(heap2, temp)
- return heap1
- otherwise
- return merge(heap2, heap1)
-
93Comparing Heaps
- Binary Heaps
- d-Heaps
- Binomial Queues
94Summary of Heap ADT Analysis
- Consider a heap of N nodes
- Space needed O(N)
- Actually, O(MaxSize) where MaxSize is the size of
the array - Pointer-based implementation pointers for
children and parent - Total space 3N 1 (3 pointers per node 1 for
size) - FindMin O(1) time DeleteMin and Insert O(log
N) time - BuildHeap from N inputs What is the run time?
- N Insert operations O(N log N)
- O(N) Treat input array as a heap and fix it
using percolate down - Thanks, Floyd!
- Mergable Heaps Binomial Queues, Leftist Heaps,
Skew Heaps