P RIORITY Q UEUES

‣ binary heaps
‣ d-ary heaps
‣ binomial heaps
‣ Fibonacci heaps

Lecture slides by Kevin Wayne

Copyright © 2005 Pearson-Addison Wesley
h t t p : / / w w w . c s . p r i n c e t o n . e d u / ~w a y n e / k l e i n b e r g - t a r d o s

Abdur Rashid Tushar

Lecturer, CSE, BUET

Last updated on 27.07.23 05:41

Priority queue data type

A min-oriented priority queue supports the following core operations:

・MAKE-HEAP(): create an empty heap.
・INSERT(H, x): insert an element x into the heap.
・EXTRACT-MIN(H): remove and return an element with the smallest key.
・DECREASE-KEY(H, x, k): decrease the key of element x to k.
The following operations are also useful:
・IS-EMPTY(H): is the heap empty?
・FIND-MIN(H): return an element with smallest key.
・DELETE(H, x): delete element x from the heap.
・MELD(H1, H2): replace heaps H1 and H2 with their union.

Note. Each element contains a key (duplicate keys are permitted)

from a totally-ordered universe.

2
Priority queue applications

Applications.
・A* search.
・Heapsort.
・Online median.
・Huffman encoding.
・Prim’s MST algorithm.
・Discrete event-driven simulation.
・Network bandwidth management.
・Dijkstra’s shortest-paths algorithm.
・…

http://younginc.site11.com/source/5895/fos0092.html
3
 P RIORITY Q UEUES

‣ binary heaps
‣ d-ary heaps
‣ binomial heaps
‣ Fibonacci heaps

SECTION 2.4
Complete binary tree

Binary tree. Empty or node with links to two disjoint binary trees.

Complete tree. Perfectly balanced, except for bottom level.

complete tree with n = 16 nodes (height = 4)

Property. Height of complete binary tree with n nodes is ⎣log2 n⎦.

Pf. Height increases (by 1) only when n is a power of 2. ▪

5
A complete binary tree in nature

6
Binary heap

Binary heap. Heap-ordered complete binary tree.

Heap-ordered tree. For each child, the key in child ≥ key in parent.

parent
1
8
0

1 1 1 2
child child
2 8 1 5

2 1 1
1 7 9

7
Explicit binary heap

Pointer representation. Each node has a pointer to parent and two children.
・Maintain number of elements n.
・Maintain pointer to root node.
・Can find pointer to last node or next node in O(log n) time.
root

1
8
0

1 1 1 2
2 8 1 5

2 1 1
1 7 9
last next

8
Implicit binary heap

Array representation. Indices start at 1.

・Take nodes in level order.
・Parent of node at k is at ⎣k / 2⎦.
・Children of node at k are at 2k and 2k + 1.
1
6

2 3
1
8
0

4 5 6 7
1 1 1 2
2 8 1 5

8 9 10
2 1 1
1 7 9

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

6 10 8 12 18 11 25 21 17 19

9
Binary heap demo

heap ordered

1
8
0

1 1 1 2
2 8 1 5

2 1 1
1 7 9

10
Binary heap: insert

Insert. Add element in new node at end; repeatedly exchange new element
with element in its parent until heap order is restored.

6
1
8
0

1 1 1 2
2 8 1 5

2 1 1 add key to heap

7
1 7 9
(violates heap order)

7 8

1 1 1 2
2 0 1 5

2 1 1 1
1 7 9 8 swim up

11
Binary heap: extract the minimum

Extract min. Exchange element in root node with last node; repeatedly
exchange element in root with its smaller child until heap order is restored.

6 element to
remove
7 8

1 1 1 2
2 0 1 5

2 1 1 1 exchange
1 7 9 8
with root

sink down

1 violates 7
8
heap order
1
7 8 8
0

1 1 1 2 1 1 1 2
2 0 1 5 2 8 1 5

2 1 1 remove 2 1 1
6 6
1 7 9 1 7 9
from heap

12
Binary heap: decrease key

Decrease key. Given a handle to node, repeatedly exchange element with its
parent until heap order is restored.

decrease key of node x to 11

1
8
0

1 1 1 2
2 8 1 5

2 1 1
1 7 9

13
Binary heap: analysis

Theorem. In an implicit binary heap, any sequence of m INSERT, EXTRACT-MIN,

and DECREASE-KEY operations with n INSERT operations takes O(m log n) time.
Pf.
・Each heap op touches nodes only on a path from the root to a leaf;
the height of the tree is at most log2 n.
・The total cost of expanding and contracting the arrays is O(n). ▪

Theorem. In an explicit binary heap with n nodes, the operations INSERT,

DECREASE-KEY, and EXTRACT-MIN take O(log n) time in the worst case.

14
Binary heap: find-min

Find the minimum. Return element in the root node.

root

1
7
0

1 1 2
8
2 1 5

2 1
9
1 7

15
Binary heap: delete

Delete. Given a handle to a node, exchange element in node with last node;
either swim down or sink up the node until heap order is restored.

delete node x or y

1
x 7
0

1 1 2
8
2 1 5
y
2 1
9
1 7
last

16
Binary heap: meld

Meld. Given two binary heaps H1 and H2, merge into a single binary heap.

Observation. No easy solution: Ω(n) time apparently required.

H1 H2
1
7
0

1 1 2
8
2 1 5

2 1
9
1 7

17
Binary heap: heapify

Heapify. Given n elements, construct a binary heap containing them.

Observation. Can do in O(n log n) time by inserting each element.

Bottom-up method. For i = n to 1, repeatedly exchange the element in node i

with its smaller child until subtree rooted at i is heap-ordered.

1 8

2 1 3 9
2

4 5 2 6 7 2
7 3
2 6

8 1 9 1 10 1 11 2
4 1 5 2

8 12 9 7 22 3 26 14 11 15 22

1 2 3 4 5 6 7 8 9 10 11
18
Binary heap: heapify

Theorem. Given n elements, can construct a binary heap containing those n

elements in O(n) time.
Pf.
・There are at most ⎡n / 2h+1⎤ nodes of height h.
・The amount of work to sink a node is proportional to its height h.
・Thus, the total work is bounded by:

Corollary. Given two binary heaps H1 and H2 containing n elements in total,

can implement MELD in O(n) time.

19
Priority queues performance cost summary

operation linked list binary heap

MAKE-HEAP O(1) O(1)

ISEMPTY O(1) O(1)

INSERT O(1) O(log n)

EXTRACT-MIN O(n) O(log n)

DECREASE-KEY O(1) O(log n)

DELETE O(1) O(log n)

MELD O(1) O(n)

FIND-MIN O(n) O(1)

20
Priority queues performance cost summary

Q. Reanalyze so that EXTRACT-MIN and DELETE take O(1) amortized time?

operation linked list binary heap binary heap †

MAKE-HEAP O(1) O(1) O(1)

ISEMPTY O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n)

EXTRACT-MIN O(n) O(log n) O(1) †

DECREASE-KEY O(1) O(log n) O(log n)

DELETE O(1) O(log n) O(1) †

MELD O(1) O(n) O(n)

FIND-MIN O(n) O(1) O(1)

† amortized
21
 P RIORITY Q UEUES

‣ binary heaps
‣ d-ary heaps
‣ binomial heaps
‣ Fibonacci heaps

SECTION 2.4
Complete d-ary tree

d-ary tree. Empty or node with links to d disjoint d-ary trees.

Complete tree. Perfectly balanced, except for bottom level.

Fact. The height of a complete d-ary tree with n nodes is ≤ ⎡logd n⎤.

23
d-ary heap

d-ary heap. Heap-ordered complete d-ary tree.

Heap-ordered tree. For each child, the key in child ≥ key in parent.

3 1 2
0 0 0

8 3 3 5 2 3 4 2 4
0 2 4 5 2 4 6 0 0

8 9
2 0

24
d-ary heap: insert

Insert. Add node at end; repeatedly exchange element in child with element
in parent until heap order is restored.

Running time. Proportional to height = O(logd n).

3 1 2
0 0 0

8 3 3 5 2 3 4 2 4
0 2 4 5 2 4 6 0 0

8 9
2 0

25
d-ary heap: extract the minimum

Extract min. Exchange root node with last node; repeatedly exchange
element in parent with element in largest child until heap order is restored.

Running time. Proportional to d ⨉ height = O(d logd n).

3 1 2
0 0 0

8 3 3 5 2 3 4 2 4
0 2 4 5 2 4 6 0 0

8 9
2 0

26
d-ary heap: decrease key

Decrease key. Given a handle to an element x, repeatedly exchange it with

its parent until heap order is restored.

Running time. Proportional to height = O(logd n).

3 1 2
0 0 0

8 3 3 5 2 3 4 2 4
0 2 4 5 2 4 6 0 0

8 9
2 0

27
Priority queues performance cost summary

operation linked list binary heap d-ary heap

MAKE-HEAP O(1) O(1) O(1)

ISEMPTY O(1) O(1) O(1)

INSERT O(1) O(log n) O(logd n)

EXTRACT-MIN O(n) O(log n) O(d logd n)

DECREASE-KEY O(1) O(log n) O(logd n)

DELETE O(1) O(log n) O(d logd n)

MELD O(1) O(n) O(n)

FIND-MIN O(n) O(1) O(1)

28
 P RIORITY Q UEUES

‣ binary heaps
‣ d-ary heaps
‣ binomial heaps
‣ Fibonacci heaps

CHAPTER 6 (2ND EDITION)

Priority queues performance cost summary

operation linked list binary heap d-ary heap

MAKE-HEAP O(1) O(1) O(1)

ISEMPTY O(1) O(1) O(1)

INSERT O(1) O(log n) O(logd n)

EXTRACT-MIN O(n) O(log n) O(d logd n)

DECREASE-KEY O(1) O(log n) O(logd n)

DELETE O(1) O(log n) O(d logd n)

MELD O(1) O(n) O(n)

FIND-MIN O(n) O(1) O(1)

Goal. O(log n) INSERT, DECREASE-KEY, EXTRACT-MIN, and MELD.

mergeable heap 30
Binomial heaps

31
Binomial tree

Def. A binomial tree of order k is defined recursively:

・Order 0: single node.
・Order k: one binomial tree of order k – 1 linked to another of order k – 1.

B0 Bk

Bk-1

Bk-1

B0 B1 B2 B3 B4
32
Binomial tree properties

Properties. Given an order k binomial tree Bk,

・Its height is k.
・It has 2k nodes.
・It has nodes at depth i.
・The degree of its root is k.
・Deleting its root yields k binomial trees Bk–1, …, B0.
Pf. [by induction on k]

Bk+1

B1 B0
B2
Bk
B4
33
Binomial heap

Def. A binomial heap is a sequence of binomial trees such that:

・Each tree is heap-ordered.
・There is either 0 or 1 binomial tree of order k.

6 3 18

8 29 10 44 37

30 23 22 48 31 17

45 32 24 50

55
B4 B1 B0
34
 Binomial heap representation

Binomial trees. Represent trees using left-child, right-sibling pointers.

Roots of trees. Connect with singly-linked list, with degrees decreasing

from left to right.

root

6 3 18 6 3 18

37
29 10 44 37 29

48 31 17 48 10

50 50 31 17 44

binomial heap leftist power-of-2 heap representation

35
Binomial heap properties

Properties. Given a binomial heap with n nodes:

・The node containing the min element is a root of B0, B1, …, or Bk.
・It contains the binomial tree Bi iff bi = 1, where bk⋅ b2 b1 b0 is binary
representation of n.
・It has ≤ ⎣log2 n⎦ + 1 binomial trees.
・Its height ≤ ⎣log2 n⎦.
6 3 18

8 29 10 44 37
n = 19
# trees = 3
30 23 22 48 31 17 height = 4
binary = 10011

45 32 24 50

55
B4 B1 B0
36
Binomial heap: meld

Meld operation. Given two binomial heaps H1 and H2, (destructively)

replace with a binomial heap H that is the union of the two.

Warmup. Easy if H1 and H2 are both binomial trees of order k.

・Connect roots of H1 and H2.
・Choose node with smaller key to be root of H.

8 29 10 44

30 23 22 48 31 17

45 32 24 50

55
H1 H2
37
 6 3 18

8 29 10 44 37

30 23 22 48 31 17
15 7 12
45 32 24 50
28 33 25

+
55

41
 12

18

6 3 18

8 29 10 44 37

30 23 22 48 31 17
15 7 12
45 32 24 50
28 33 25

+
55

41
 3 12

7 37 18

25

6 3 18

8 29 10 44 37

30 23 22 48 31 17
15 7 12
45 32 24 50
28 33 25

+
55

41

12

18
 3 3 12

15 7 37 7 37 18

28 33 25 25

41
6 3 18

8 29 10 44 37

30 23 22 48 31 17
15 7 12
45 32 24 50
28 33 25

+
55

41

12

18
 3 3 12

15 7 37 7 37 18

28 33 25 25

41
6 3 18

8 29 10 44 37

30 23 22 48 31 17
15 7 12
45 32 24 50
28 33 25

+
55

41

3 12

15 7 37 18

28 33 25

41
 3 3 12

15 7 37 7 37 18

28 33 25 25

41
6 3 18

8 29 10 44 37

30 23 22 48 31 17
15 7 12
45 32 24 50
28 33 25

+
55

41

6 3 12

8 29 10 44 15 7 37 18

30 23 22 48 31 17 28 33 25

45 32 24 50 41

55
 6 3 18

8 29 10 44 37

30 23 22 48 31 17
15 7 12
45 32 24 50
28 33 25

+
55

41

1 1 1

1 0 0 1 1
19 + 7 = 26 + 0 0 1 1 1

1 1 0 1 0
Binomial heap: meld

Meld operation. Given two binomial heaps H1 and H2, (destructively)

replace with a binomial heap H that is the union of the two.

Solution. Analogous to binary addition.

Running time. O(log n).

Pf. Proportional to number of trees in root lists ≤ 2 ( ⎣log2 n⎦ + 1). ▪

1 1 1

1 0 0 1 1
19 + 7 = 26 + 0 0 1 1 1

1 1 0 1 0

45
Binomial heap: extract the minimum

Extract-min. Delete the node with minimum key in binomial heap H.

・Find root x with min key in root list of H, and delete.

3 6 18

8 29 10 44 37

30 23 22 48 31 17
H

45 32 24 50

55
46
Binomial heap: extract the minimum

Extract-min. Delete the node with minimum key in binomial heap H.

・Find root x with min key in root list of H, and delete.
・H′ ← broken binomial trees.
・H ← MELD(H′, H).
Running time. O(log n).

6 18

8 29 10 44 37

30 23 22 48 31 17
H

45 32 24 50
H′
55
47
Binomial heap: decrease key

Decrease key. Given a handle to an element x in H, decrease its key to k.

・Suppose x is in binomial tree Bk.
・Repeatedly exchange x with its parent until heap order is restored.

Running time. O(log n).

3 6 18

8 29 10 44 37

30 23 22 48 31 17
H

x 32 24 50

55
48
Binomial heap: delete

Delete. Given a handle to an element x in a binomial heap, delete it.

・DECREASE-KEY(H, x, -∞).
・DELETE-MIN(H).
Running time. O(log n).

3 6 18

8 29 10 44 37

30 23 22 48 31 17
H

45 32 24 50

55
49
Binomial heap: insert

Insert. Given a binomial heap H, insert an element x.

・H′ ← MAKE-HEAP( ).
・H′ ← INSERT(H′, x).
・H ← MELD(H′, H).

Running time. O(log n).

3 6 18 x

8 29 10 44 H′
37

30 23 22 48 31 17
H

45 32 24 50

55
50
Binomial heap: sequence of insertions

Insert. How much work to insert a new node x ?

・If n = .......0, then only 1 credit. 3 6 x

・If n = .......01, then only 2 credits.

・If n = .......011, then only 3 credits. 29 10 44 37

・If n = .......0111, then only 4 credits. 48 31 17

50

Observation. Inserting one element can take Ω(log n) time.

if n = 11...111

Theorem. Starting from an empty binomial heap, a sequence of n

consecutive INSERT operations takes O(n) time.

Pf. (n / 2) (1) + (n / 4)(2) + (n / 8)(3) + … ≤ 2 n. ▪

51
Binomial heap: amortized analysis

Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the
worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n).

Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.
・Φ(H0) = 0.
・Φ(Hi) ≥ 0 for each binomial heap Hi.
Case 1. [INSERT]
・Actual cost ci = number of trees merged + 1.
・∆Φ = Φ(Hi) – Φ(Hi–1) = 1 – number of trees merged.
・Amortized cost = ĉi = ci + Φ(Hi) – Φ(Hi–1) = 2.

52
Binomial heap: amortized analysis

Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the
worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n).

Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.
・Φ(H0) = 0.
・Φ(Hi) ≥ 0 for each binomial heap Hi.
Case 2. [ DECREASE-KEY ]
・Actual cost ci = O(log n).
・∆Φ = Φ(Hi) – Φ(Hi–1) = 0.
・Amortized cost = ĉi = ci = O(log n).

53
Binomial heap: amortized analysis

Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the
worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n).

Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.
・Φ(H0) = 0.
・Φ(Hi) ≥ 0 for each binomial heap Hi.
Case 3. [ EXTRACT-MIN or DELETE ]
・Actual cost ci = O(log n).
・∆Φ = Φ(Hi) – Φ(Hi–1) ≤ Φ(Hi) ≤ ⎣log2 n⎦.
・Amortized cost = ĉi = ci + Φ(Hi) – Φ(Hi–1) = O(log n). ▪

54
Priority queues performance cost summary

operation linked list binary heap binomial heap binomial heap

MAKE-HEAP O(1) O(1) O(1) O(1)

ISEMPTY O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1) †

EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(log n)

DELETE O(1) O(log n) O(log n) O(log n)

homework
MELD O(1) O(n) O(log n) O(1) †

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

Hopeless challenge. O(1) INSERT, DECREASE-KEY and EXTRACT-MIN. Why?

Challenge. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

55