CSE 326: Data Structures Priority Queues - Binary Heaps
CSE 326: Data Structures Priority Queues - Binary Heaps
1
Recall Queues
• FIFO: First-In, First-Out
6 2
15 23
insert deleteMin
12 18
45 3 7
3
Priority Queue ADT
1. PQueue data : collection of data with
priority
2. PQueue operations
– insert
– deleteMin
• Anything greedy
5
Potential Implementations
insert deleteMin
Unsorted list (Array) O(1) O(n)
Unsorted list (Linked-List) O(1) O(n)
6
Recall From Lists, Queues,
Stacks
• Use an ADT that corresponds to your
needs
• The right ADT is efficient, while an overly
general ADT provides functionality you
aren’t using, but are paying for anyways
• Heaps provide O(log n) worst case for
both insert and deleteMin, O(1) average
insert
7
Binary Heap Properties
1. Structure Property
2. Ordering Property
8
Tree Review Tree T
A
root(T):
leaves(T): B C
children(B):
parent(H): D E F G
siblings(E):
H I
ancestors(F):
descendents(G):
subtree(C): J K L M N
9
More Tree Terminology Tree T
A
depth(B):
height(G): B C
degree(B): D E F G
branching factor(T):
H I
J K L M N
10
Brief interlude: Some Definitions:
A Perfect binary tree – A binary tree with
all leaf nodes at the same depth. All
internal nodes have 2 children.
height h
11 2h+1 – 1 nodes
2h – 1 non-leaves
5 21 2h leaves
2 9 16 25
1 3 7 10 13 19 22 30
11
Heap Structure Property
• A binary heap is a complete binary tree.
Complete binary tree – binary tree that is
completely filled, with the possible exception
of the bottom level, which is filled left to right.
Examples:
12
Representing Complete
Binary Trees in an Array
1 A From node i:
2 3
B C
4
D 5
E
6
F
7
G
left child:
8 9 10 11 12 right child:
H I J K L
parent:
13
Why this approach to storage?
14
Heap Order Property
Heap order property: For every non-root
node X, the value in the parent of X is
less than (or equal to) the value in X.
10
10
20 80
20 80
40 60 85 99
30 15
50 700
not a heap
15
Heap Operations
• findMin:
• insert(val): percolate up.
• deleteMin: percolate down.
10
20 80
40 60 85 99
50 700 65
16
Heap – Insert(val)
Basic Idea:
1. Put val at “next” leaf position
2. Percolate up by repeatedly
exchanging node until no longer
needed
17
Insert: percolate up
10
20 80
40 60 85 99
50 700 65 15
10
15 80
40 20 85 99
50 700 65 60
18
Insert Code (optimized)
void insert(Object o) { int percolateUp(int hole,
Object val) {
assert(!isFull());
while (hole > 1 &&
size++; val < Heap[hole/2])
newPos = Heap[hole] = Heap[hole/2];
hole /= 2;
percolateUp(size,o);
}
Heap[newPos] = o; return hole;
} }
runtime:
(Code in book) 19
Heap – Deletemin
Basic Idea:
1. Remove root (that is always the min!)
2. Put “last” leaf node at root
3. Find smallest child of node
4. Swap node with its smallest child if needed.
5. Repeat steps 3 & 4 until no swaps needed.
20
DeleteMin: percolate down
10
20 15
40 60 85 99
50 700 65
15
20 65
40 60 85 99
50 700
21
DeleteMin Code (Optimized)
Object deleteMin() { int percolateDown(int hole,
Object val) {
assert(!isEmpty());
while (2*hole <= size) {
returnVal = Heap[1]; left = 2*hole;
size--; right = left + 1;
if (right ≤ size &&
newPos =
Heap[right] < Heap[left])
percolateDown(1, target = right;
Heap[size+1]); else
target = left;
Heap[newPos] =
Heap[size + 1]; if (Heap[target] < val) {
return returnVal; Heap[hole] = Heap[target];
hole = target;
}
}
else
runtime: break;
}
(code in book) return hole; 22
}
Insert: 16, 32, 4, 69, 105, 43, 2
0 1 2 3 4 5 6 7 8
23
Data Structures
Binary Heaps
24
Building a Heap
12 5 11 3 10 6 9 4 8 1 7 2
25
Building a Heap
• Adding the items one at a time is O(n
log n) in the worst case
26
Working on Heaps
• What are the two properties of a heap?
– Structure Property
– Order Property
27
BuildHeap: Floyd’s Method
12 5 11 3 10 6 9 4 8 1 7 2
12
5 11
3 10 6 9
4 8 1 7 2 28
Buildheap pseudocode
private void buildHeap() {
for ( int i = currentSize/2; i > 0; i-- )
percolateDown( i );
}
runtime:
29
BuildHeap: Floyd’s Method
12
5 11
3 10 6 9
4 8 1 7 2
30
BuildHeap: Floyd’s Method
12
5 11
3 10 2 9
4 8 1 7 6
31
BuildHeap: Floyd’s Method
12 12
5 11 5 11
3 10 2 9 3 1 2 9
4 8 1 7 6 4 8 10 7 6
32
BuildHeap: Floyd’s Method
12 12
5 11 5 11
3 10 2 9 3 1 2 9
4 8 1 7 6 4 8 10 7 6
12
5 2
3 1 6 9
33
4 8 10 7 11
BuildHeap: Floyd’s Method
12 12
5 11 5 11
3 10 2 9 3 1 2 9
4 8 1 7 6 4 8 10 7 6
12 12
5 2 1 2
3 1 6 9 3 5 6 9
34
4 8 10 7 11 4 8 10 7 11
Finally…
1
3 2
4 5 6 9
12 8 10 7 11
runtime:
35
More Priority Queue
• decreaseKey
Operations
– given a pointer to an object in the queue, reduce its priority value
• FindMax
37
Facts about Heaps
Observations:
• Finding a child/parent index is a multiply/divide by two
• Operations jump widely through the heap
• Each percolate step looks at only two new nodes
• Inserts are at least as common as deleteMins
Realities:
• Division/multiplication by powers of two are equally fast
• Looking at only two new pieces of data: bad for cache!
• With huge data sets, disk accesses dominate
38
Cycles to access:
CPU
Cache
Memory
Disk
39
A Solution: d-Heaps
• Each node has d
children 1
• Still representable by
array 3 7 2
• Good choices for d:
– (choose a power of two 4 8 5 12 11 10 6 9
for efficiency)
– fit one set of children in a 12 1 3 7 2 4 8 5 12 11 10 6 9
cache line
– fit one set of children on a
memory page/disk block
40
One More Operation
• Merge two heaps
41
CSE 326: Data Structures
Priority Queues
Leftist Heaps & Skew Heaps
42
New Heap Operation: Merge
Given two heaps, merge them into one heap
– first attempt: insert each element of the
smaller heap into the larger.
runtime:
43
Leftist Heaps
Idea:
Focus all heap maintenance work in
one small part of the heap
Leftist heaps:
1. Most nodes are on the left
2. All the merging work is done on the right
44
Definition: Null Path Length
null path length (npl) of a node x = the number of nodes between x
and a null in its subtree
OR
npl(x) = min distance to a descendant with 0 or 1 children
• npl(null) = -1 ?
• npl(leaf) = 0
• npl(single-child node) = 0 ? ?
Equivalent definitions: 0 1 ? 0
• Leftist property
– For every node x, npl(left(x)) npl(right(x))
– result: tree is at least as “heavy” on the left as the right
46
Are These Leftist?
2 2 0
1 1 1 1 0
0 1 0 0 1 0 0 0 1
0 0 0 0 0 0 0 0
0
Every subtree of a leftist
0
tree is leftist!
0
47
Right Path in a Leftist Tree is Short (#1)
Claim: The right path is as short as any in the tree.
Proof: (By contradiction)
51
Merging Two Leftist Heaps
• merge(T1,T2) returns one leftist heap containing all
elements of the two (distinct) leftist heaps T 1 and T2
merge
T1 a a
merge
L1 R1 L1 R1
a<b
T2 b b
L2 R2 L2 R2 52
Merge Continued
a a
If npl(R’) > npl(L1)
L1 R’ R’ L1
R’ = Merge(R1, T2)
runtime:
53
Let’s do an example, but first…
Other Heap Operations
• insert ?
• deleteMin ?
54
Operations on Leftist Heaps
• merge with two trees of total size n: O(log n)
• insert with heap size n: O(log n)
– pretend node is a size 1 leftist heap
– insert by merging original heap with one node
heap
merge
merge
55
Leftest Merge Example
merge
?
1 3
5
0 0
0 merge
10 12 7 1 ?
5 5
0
1 14
3 0 0 0 merge
10 12 10 0
0 0 12
7 8 0 0
8 8
0
14
0
(special case) 8
0
12
56
Sewing Up the Example
? ? 1
3 3 3
0 0 0
7 ? 7 1 7 5 1
5 5
0 0 0 0 0
14 0 0 0
10 0 14 14 10 8
8 10 8
0
0 0 12
12 12
Done?
57
Finally…
1 1
3 3
0 0
7 5 1 5 1 7
0 0 0 0 0 0
14 10 8 10 8 14
0 0
12 12
58
Leftist Heaps: Summary
Good
•
•
Bad
•
•
59
Random Definition:
Amortized Time
am·or·tized time:
Running time limit resulting from “writing off” expensive
runs of an algorithm over multiple cheap runs of the
algorithm, usually resulting in a lower overall running time
than indicated by the worst possible case.
If M operations take total O(M log N) time,
amortized time per operation is O(log N)
60
Skew Heaps
Problems with leftist heaps
– extra storage for npl
– extra complexity/logic to maintain and check npl
– right side is “often” heavy and requires a switch
Solution: skew heaps
– “blindly” adjusting version of leftist heaps
– merge always switches children when fixing right
path
– amortized time for: merge, insert, deleteMin = O(log
n)
– however, worst case time for all three = O(n)
61
Merging Two Skew Heaps
merge
T1 a a
merge
L1 R1 R1 L1
a<b
T2 b b
L2 R2 L2 R2
8 10 14
63
12
Skew Heap Code
void merge(heap1, heap2) {
case {
heap1 == NULL: return heap2;
heap2 == NULL: return heap1;
heap1.findMin() < heap2.findMin():
temp = heap1.right;
heap1.right = heap1.left;
heap1.left = merge(heap2, temp);
return heap1;
otherwise:
return merge(heap2, heap1);
}
}
64
Runtime Analysis:
Worst-case and Amortized
• No worst case guarantee on right path
length!
• All operations rely on merge
67
Yet Another Data Structure:
Binomial Queues
• Structural property
– Forest of binomial trees with at most
one tree of any height
What’s a forest?
• Order property
– Each binomial tree has the heap-order
property
68
The Binomial Tree, Bh
• Bh has height h and exactly 2h nodes
• Bh is formed by making Bh-1 a child of another Bh-1
• Root has exactly h children
• Number of nodes at depth d is binomial coeff.
– Hence the name; we will not use this last property
h
d
B0 B1 B2 B3
69
Binomial Queue with n elements
Binomial Q with n elements has a unique structural
representation in terms of binomial trees!
1 B3 1 B2 No B1 1 B0
70
Properties of Binomial Queue
• At most one binomial tree of any height
71
Operations on Binomial Queue
• Will again define merge as the base operation
– insert, deleteMin, buildBinomialQ will use merge
72
Merging Two Binomial Queues
Essentially like adding two binary numbers!
1. Combine the two forests
2. For k from 0 to maxheight {
a. m total number of Bk’s in the two BQs
b. if m=0: continue; # of 1’s
c. if m=1: continue; 0+0 = 0
d. if m=2: combine the two Bk’s to form a1+0 = 1
Bk+1 1+1 = 0+c
e. if m=3: retain one Bk and 1+1+c = 1+c
combine the other two to form a Bk+1
}
Claim: When this process ends, the forest
has at most one tree of any height 73
Example: Binomial Queue
Merge
H1: H2:
21 1 -1 3 5
7 2 1 3 9 6
8 11 5 7
74
Example: Binomial Queue
Merge
H1: H2:
1 -1 3 5
7 2 1 3 21 9 6
8 11 5 7
75
Example: Binomial Queue
Merge
H1: H2:
1 -1 5
7 3 2 1 3 9 6
21 8 11 5 7
76
Example: Binomial Queue
Merge
H1: H2:
1 -1
7 3 5 2 1 3
21 9 6 8 11 5
7 6
77
Example: Binomial Queue
Merge
H1: H2:
-1
2 1 3 1
8 11 5 7 3 5
6 21 9 6
78
Example: Binomial Queue
Merge
H1: H2:
-1
2 1 3 1
8 11 5 7 3 5
6 21 9 6
79
Complexity of Merge
Constant time for each height
Max number of heights is: log n
80
Insert in a Binomial Queue
Insert(x): Similar to leftist or skew heap
runtime
Worst case complexity: same as merge
O( )
81
deleteMin in Binomial Queue
Similar to leftist and skew heaps….
82
deleteMin: Example
BQ 7
3
4
8 5
8 5
runtime:
84