Priority Queues and Heaps

Outline and Reading

• PriorityQueue ADT (§8.1)
• Total order relation (§8.1.1)
• Comparator ADT (§8.1.2)
• Sorting with a Priority Queue (§8.1.5)
• Implementing a PQ with a list (§8.2)
• Selection-sort and Insertion Sort (§8.2.2)
• Heaps (§8.3)
• Complete Binary Trees (§8.3.2)
• Implementing a PQ with a heap (§8.3.3)
• Heapsort (§8.3.5)
 Priority Queue ADT
• A priority queue stores a • Additional methods
collection of items • minKey()
returns, but does not remove,
• An item is a pair the smallest key of an item
(key, element) • minElement()
• Main methods of the Priority returns, but does not remove,
Queue ADT the element of an item with
smallest key
• insertItem(k, o)
• size(), isEmpty()
inserts an item with key k and
element o • Applications:
• removeMin() • Standby flyers
removes the item with the • Auctions
smallest key
 Comparator ADT
• A comparator encapsulates the action of comparing
two objects according to a given total order relation
• A generic priority queue uses a comparator as a
template argument, to define the comparison function
(<,=,>)
• The comparator is external to the keys being compared.
Thus, the same objects can be sorted in different ways
by using different comparators.
• When the priority queue needs to compare two keys, it
uses its comparator
 Using Comparators in C++
• A comparator class overloads the “()” operator with a comparison
function.
• Example: Compare two points in the plane lexicographically.

class LexCompare {
public:
int operator()(Point a, Point b) {
if (a.x < b.x) return –1;
else if (a.x > b.x) return +1;
else if (a.y < b.y) return –1;
else if (a.y > b.y) return +1;
else return 0;
}
};
 Total Order Relation

• Keys in a priority • Mathematical concept of

queue can be arbitrary total order relation 
objects on which an
• Reflexive property:
order is defined
xx
• Two distinct items in a
• Antisymmetric property:
priority queue can
xyyxx=y
have the same key
• Transitive property:
xyyzxz
 PQ-Sort:
Sorting with a Priority Queue

• We can use a priority queue

to sort a set of comparable Algorithm PQ-Sort(S, C)
elements Input sequence S, comparator C for the
• Insert the elements one by elements of S
one with a series of Output sequence S sorted in increasing
order according to C
insertItem(e, e) operations
P  priority queue with
• Remove the elements in comparator C
sorted order with a series of while !S.isEmpty ()
removeMin() operations
e  S.remove (S.first())
P.insertItem(e, e)
• Running time depends on the while !P.isEmpty()
PQ implementation e  P.minElement()
P.removeMin()
S.insertLast(e)
 List-based Priority Queue
• Unsorted list implementation • sorted list implementation
• Store the items of the priority queue in • Store the items of the priority
a list-based sequence, in arbitrary order queue in a sequence, sorted by
key
4 5 2 3 1 1 2 3 4 5
• Performance:
• insertItem takes O(1) time since we
• Performance:
can insert the item at the beginning • insertItem takes O(n) time
or end of the sequence since we have to find the place
• removeMin, minKey and where to insert the item
minElement take O(n) time since • removeMin, minKey and
we have to traverse the entire minElement take O(1) time
sequence to find the smallest key since the smallest key is at the
beginning of the sequence
 Selection-Sort
• Selection-sort is the variation of PQ-sort
where the priority queue is implemented
with an unsorted sequence
4 5 2 3 1

• Running time of Selection-sort:

• Inserting the elements into the priority queue
with n insertItem operations takes O(n) time
• Removing the elements in sorted order from the
priority queue with n removeMin operations
takes time proportional to
1 2 …n
• Selection-sort runs in O(n2) time
 Exercise: Selection-Sort

• Selection-sort is the variation of PQ-sort

where the priority queue is implemented
with an unsorted sequence (first n
insertItems, then n removeMins)
4 5 2 3 1

• Illustrate the performance of selection-

sort on the following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)
 Insertion-Sort
• Insertion-sort is the variation of PQ-sort where the priority queue
is implemented with a sorted sequence
1 2 3 4 5
• Running time of Insertion-sort:
• Inserting the elements into the priority queue with n insertItem
operations takes time proportional to
1 2 …n
• Removing the elements in sorted order from the priority queue with
a series of n removeMin operations takes O(n) time
• Insertion-sort runs in O(n2) time
 Exercise: Insertion-Sort

• Insertion-sort is the variation of PQ-sort

where the priority queue is implemented
with a sorted sequence (first n insertItems,
then n removeMins)
1 2 3 4 5

• Illustrate the performance of insertion-

sort on the following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)
 In-place Insertion-sort
• Instead of using an external
5 4 2 3 1
data structure, we can
implement selection-sort
and insertion-sort in-place 5 4 2 3 1
(only O(1) extra storage)
• A portion of the input 4 5 2 3 1
sequence itself serves as the
priority queue 2 4 5 3 1
• For in-place insertion-sort
• We keep sorted the initial 2 3 4 5 1
portion of the sequence
• We can use swapElements
instead of modifying the 1 2 3 4 5
sequence
1 2 3 4 5
Heaps and Priority Queues

5 6

9 7
 What is a heap?
• A heap is a binary tree storing • The last node of a heap is
keys at its internal nodes and the rightmost internal
satisfying the following node of depth h  1
properties:
• Heap-Order: for every internal
2
node v other than the root,
key(v)  key(parent(v))
5 6
• Complete Binary Tree: let h be
the height of the heap
• for i 0, … , h 1, there are 9 7
2i nodes of depth i
• at depth h  1, the internal
nodes are to the left of the leaf
nodes
last node
 Height of a Heap
• Theorem: A heap storing n keys has height O(log n)
• Proof: (we apply the complete binary tree property)
• Let h be the height of a heap storing n keys
• Since there are 2i keys at depth i  0, … , h 2 and at least one key at
depth h 1, we have n  1 2  4  …  2h2 1
• Thus, n  2h1 , i.e., h  log n 1

depth keys
0 1

1 2
h2 2h2
h1 1
 Exercise: Heaps
• Where may an item with the largest key be stored
in a heap?

• True or False:
• A pre-order traversal of a heap will list out its keys in
sorted order. Prove it is true or provide a counter
example.

• Let H be a heap with 7 distinct elements

(1,2,3,4,5,6, and 7). Is it possible that a preorder
traversal visits the elements in sorted order? What
about an inorder traversal or a postorder
traversal? In each case, either show such a heap or
prove that none exists.
 Heaps and Priority Queues
• We can use a heap to implement a priority queue
• We store a (key, element) item at each internal node
• We keep track of the position of the last node
• For simplicity, we show only the keys in the pictures

(2, Sue)

(5, Pat) (6, Mark)

(9, Jeff) (7, Anna)

Insertion into a Heap
• Method insertItem of the 2
priority queue ADT corresponds
to the insertion of a key k to the 5 z 6
heap
• The insertion algorithm consists 9 7
of three steps
• Find the insertion node z (the insertion node
new last node)
• Store k at z and expand z into 2
an internal node
• Restore the heap-order 5 6
property (discussed next) z
9 7 1
 Upheap
• After the insertion of a new key k, the heap-order property may be
violated
• Algorithm upheap restores the heap-order property by swapping k along
an upward path from the insertion node
• Upheap terminates when the key k reaches the root or a node whose
parent has a key smaller than or equal to k
• Since a heap has height O(log n), upheap runs in O(log n) time

2 1

5 1 5 2
z z
9 7 6 9 7 6
 Removal from a Heap
• Method removeMin of 2
the priority queue ADT
corresponds to the 5 6
removal of the root key w
from the heap 9 7
• The removal algorithm
consists of three steps last node
• Replace the root key
with the key of the last 7
node w
• Compress w and its 5
w
6
children into a leaf
9
• Restore the heap-order
property (discussed
next)
 Downheap
• After replacing the root key with the key k of the last node, the heap-
order property may be violated
• Algorithm downheap restores the heap property by swapping key k with
the child with the smallest key along a downward path from the root
• Downheap terminates when key k reaches a leaf or a node whose
children have keys greater than or equal to k
• Since a heap has height O(log n), downheap runs in O(log n) time

7 5

5 6 7 6
w w
9 9
 Updating the Last Node
• The insertion node can be found by traversing a path of O(log n)
nodes
• Go up until a left child or the root is reached
• If a left child is reached, go to the right child
• Go down left until a leaf is reached
• Similar algorithm for updating the last node after a removal
 Heap-Sort

• Consider a priority queue • Using a heap-based

with n items implemented priority queue, we can
by means of a heap sort a sequence of n
• the space used is O(n) elements in O(n log n)
• methods insertItem and time
removeMin take O(log n) • The resulting algorithm is
time called heap-sort
• methods size, isEmpty, • Heap-sort is much faster
minKey, and minElement than quadratic sorting
take time O(1) time algorithms, such as
insertion-sort and
selection-sort
 Exercise: Heap-Sort

• Heap-sort is the variation of PQ-sort

where the priority queue is implemented
with a heap (first n insertItems, then n
removeMins)
4 5 2 3 1

• Illustrate the performance of heap-sort on

the following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)
Vector-based Heap Implementation
• We can represent a heap with n keys
by means of a vector of length n 2
• For the node at rank i
• the left child is at rank 2i+1 5 6
• the right child is at rank 2i +2
i 1
• parent is  2  9 7
 
• Links between nodes are not
explicitly stored
• The leaves are not represented
• Operation insertItem corresponds to
inserting at rank n 2 5 6 9 7
• Operation removeMin corresponds 0 1 2 3 4
to removing at rank n-1
• Yields in-place heap-sort
 Priority Queue Sort Summary
• PQ-Sort consists of n insertions followed by n
removeMin ops
Insert RemoveMin PQ-Sort
Total

Insertion Sort (ordered O(n) O(1) O(n2)

sequence)

Selection Sort O(1) O(n) O(n2)

(unordered sequence)
Heap Sort (binary heap, O(logn) O(logn) O(n logn)
vector-based
implementation)
 Merging Two Heaps
3 2
• We are given two two
8 5 4 6
heaps and a key k
• We create a new heap
with the root node 7

storing k and with the 3 2

two heaps as subtrees 8 5 4 6
• We perform
downheap to restore 2
the heap-order
3 4
property
8 5 7 6
Bottom-up Heap Construction
• We can construct a
heap storing n given
keys in using a 2i 1 2i 1
bottom-up
construction with log
n phases
• In phase i, pairs of
heaps with 2i 1 keys
are merged into 2i11
heaps with 2i11
keys
 Example

16 15 4 12 6 9 23 20
 Example

25 5 11 27

16 15 4 12 6 9 23 20
 Example

15 4 6 20

16 25 5 12 11 9 23 27
 Example

7 8

15 4 6 20

16 25 5 12 11 9 23 27
 Example

4 6

15 5 8 20

16 25 7 12 11 9 23 27
 Example
10

4 6

15 5 8 20

16 25 7 12 11 9 23 27
 Example
4

5 6

15 7 8 20

16 25 10 12 11 9 23 27
 Analysis
• At each level we insert 2i nodes
• Each node can generate h-i swaps
 Analysis
• At each level we insert 2i nodes
• Each node can generate h-i swaps

2i
 Analysis
• At each level we insert 2i nodes
• Each node can generate h-i swaps

hi
 Analysis
• At each level we insert 2i nodes
• Each node can generate h-i swaps

h 1

 2 (h  i )
i 0
i
 Analysis
• At each level we insert 2i nodes
• Each node can generate h-i swaps

h 1

 2 (h  i )  2
i 0
i h 1
 (h  2)
 Analysis
• At each level we insert 2i nodes
• Each node can generate h-i swaps

h 1

 2 (h  i)  2n  (log(n)  2)
i 0
i
 Analysis
• At each level we insert 2i nodes
• Each node can generate h-i swaps
• Thus, bottom-up heap construction runs in O(n) time
• Bottom-up heap construction is faster than n successive
insertions and speeds up the first phase of heap-sort
 Exercise: Bottom-up Heap

• Build a heap from the bottom up on the

following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25, 2, 11, 7, 1, 17)
Locators 3 a

1 g 4 e
 Locators
• A locator identifies and tracks an item (key, element) within
a data structure
• A locator sticks with a specific item, even if that element
changes its position in the data structure
• Intuitive notion:
• claim check
• reservation number
• Methods of the locator ADT:
• key(): returns the key of the item associated with the locator
• element(): returns the element of the item associated with the
locator
 Locator-based Methods
• Locator-based priority queue Locator-based dictionary
methods: methods:
• insert(k, o): inserts the item (k, • insert(k, o): inserts the item (k,
o) and returns a locator for it o) and returns its locator
• min(): returns the locator of an • find(k): if the dictionary contains
item with smallest key an item with key k, returns its
• remove(l): remove the item locator, else return a special null
with locator l locator
• replaceKey(l, k): replaces the • remove(l): removes the item
key of the item with locator l with locator l and returns its
• replaceElement(l, o): replaces element
with o the element of the item • locators(), replaceKey(l, k),
with locator l replaceElement(l, o)
• locators(): returns an iterator
over the locators of the items
in the priority queue
 Possible Implementation
• The locator is an 6 d
object storing
• key 3 a 9 b
• element
• position (or rank) of
the item in the
underlying structure
• In turn, the position
(or array cell) stores
the locator
• Example:
• binary search tree
with locators
1 g 4 e 8 c
 Positions vs. Locators
• Position • Locator
• represents a “place” in a data • identifies and tracks a (key,
structure element) item
• related to other positions in • unrelated to other locators in
the data structure (e.g., the data structure
previous/next or parent/child) • often implemented as an object
• often implemented as a storing the item and its position
pointer to a node or the index in the underlying structure
of an array cell • Key-based ADTs (e.g., priority
• Position-based ADTs (e.g., queue and dictionary) can be
sequence and tree) are augmented with locator-based
fundamental data storage methods
schemes