Priority Queues

Binary and Binomial Heaps

Supports the following operations.

Insert element x.

Return min element.

Return and delete minimum element.

Decrease key of element x to k.

Applications.

These lecture slides are adapted

from CLRS, Chapters 6, 19.

Dijkstras shortest path algorithm.

Prims MST algorithm.

Event-driven simulation.

Huffman encoding.

Heapsort.

...

Princeton University COS 423 Theory of Algorithms Spring 2002 Kevin Wayne

Priority Queues in Action

Priority Queues
Heaps

Dijkstras Shortest Path Algorithm

PQinit()
for each v V
key(v)
PQinsert(v)
key(s) 0
while (!PQisempty())
v = PQdelmin()
for each w Q s.t (v,w) E
if (w) > (v) + c(v,w)
PQdecrease(w, (v) + c(v,w))

Operation

Linked List

Binary

Binomial

Fibonacci *

Relaxed

make-heap

insert

log N

log N

find-min

log N

delete-min

log N

log N

log N

log N

union

log N

decrease-key

log N

log N

delete

log N

log N

log N

log N

is-empty

Dijkstra/Prim
1 make-heap
|V| insert
|V| delete-min
|E| decrease-key
3

O(|V|2)

O(|E| log |V|)

O(|E| + |V| log |V|)

Binary Heap: Definition

Binary Heap: Properties

Binary heap.

Properties.

Almost complete binary tree.

filled on all levels, except last, where filled from left to right

Min element is in root.

Heap with N elements has height = log2 N.

Min-heap ordered.
every child greater than (or equal to) parent
06

06

14

78

91

83

14

45

18

47

77

81

84

78

53

99

N = 14
Height = 3

18

91

83

64

45

47

77

81

84

53

99

64

Binary Heaps: Array Implementation

Binary Heap: Insertion

Implementing binary heaps.

Insert element x into heap.

Use an array: no need for explicit parent or child pointers.

Parent(i) = i/2
Left(i)
= 2i
Right(i)
= 2i + 1

Insert into next available slot.

Bubble up until its heap ordered.
Peter principle: nodes rise to level of incompetence

06

06

1
14

45

14

78

18

47

53

83

91

81

10

77

84

99

64

11

12

13

14

78

83

45

47

18

91

81

77

84

53

99

64

42

next free slot

8

Binary Heap: Insertion

Binary Heap: Insertion

Insert element x into heap.

Insert element x into heap.

Insert into next available slot.

Bubble up until its heap ordered.

Peter principle: nodes rise to level of incompetence

14

Bubble up until its heap ordered.

Peter principle: nodes rise to level of incompetence

14

47

77

81

84

78

53

99

swap with parent

06

45

18

91

83

Insert into next available slot.

swap with parent

06

78

64

18

91

83

42

45

47

77

81

84

42

99

64

53
42

10

Binary Heap: Insertion

Binary Heap: Decrease Key

Insert element x into heap.

Decrease key of element x to k.

Insert into next available slot.

Bubble up until its heap ordered.
Peter principle: nodes rise to level of incompetence

Bubble up until its heap ordered.

O(log N) operations.

O(log N) operations.
stop: heap ordered

06
14

78

83

81

14

42

47

18

91

06

77

84

78

45

99

64

83

53

11

42

47

18

91

81

77

84

45

99

64

53

12

Binary Heap: Delete Min

Binary Heap: Delete Min

Delete minimum element from heap.

Delete minimum element from heap.

Exchange root with rightmost leaf.

Bubble root down until its heap ordered.

power struggle principle: better subordinate is promoted

Exchange root with rightmost leaf.

Bubble root down until its heap ordered.
power struggle principle: better subordinate is promoted

06

53

14

78

18

91

83

14

42

47

77

81

84

78

45

99

64

18

91

83

53

42

47

77

81

84

45

99

64

06

13

14

Binary Heap: Delete Min

Binary Heap: Delete Min

Delete minimum element from heap.

Delete minimum element from heap.

Exchange root with rightmost leaf.

Bubble root down until its heap ordered.

power struggle principle: better subordinate is promoted

14

83

81

53

47

77

84

78

45

99

exchange with right child

14

42

18

91

Bubble root down until its heap ordered.

power struggle principle: better subordinate is promoted

exchange with left child

53

78

Exchange root with rightmost leaf.

83

64

15

42

47

18

91

81

77

84

45

99

64

16

Binary Heap: Delete Min

Binary Heap: Heapsort

Delete minimum element from heap.

Heapsort.

Exchange root with rightmost leaf.

Bubble root down until its heap ordered.
power struggle principle: better subordinate is promoted
O(log N) operations.

18

Perform N delete-min operations.

O(N log N) sort.

No extra storage.

42

78

53

91

Insert N items into binary heap.

stop: heap ordered

14

83

47

77

81

84

45

99

64

17

18

Binary Heap: Union

Priority Queues
Heaps

Union.

Combine two binary heaps H1 and H2 into a single heap.

Operation

Linked List

Binary

Binomial

Fibonacci *

Relaxed

No easy solution.
(N) operations apparently required

make-heap

insert

log N

log N

find-min

log N

Can support fast union with fancier heaps.

delete-min

log N

log N

log N

log N

H1

H2

union

log N

14

11

decrease-key

log N

log N

delete

log N

log N

log N

log N

is-empty

78

41

18

91

81

53

77

84

62

99

64

19

20

Binomial Tree

Binomial Tree

Binomial tree.

Useful properties of order k binomial tree Bk.

Recursive definition:
Bk

B0

Number of nodes = 2k.

Height = k.

Degree of root = k.

Bk-1

Bk+1

Deleting root yields binomial

trees Bk-1, , B0.

Bk-1

Bk

Proof.

B0

B1

B2

B3

B0

B4

B2

B1

B0

By induction on k.

B1

B2

B3

B4

21

22

Binomial Tree

Binomial Heap

A property useful for naming the data structure.

Bk has

k

i

Binomial heap. Vuillemin, 1978.

nodes at depth i.

Sequence of binomial trees that satisfy binomial heap property.

each tree is min-heap ordered
0 or 1 binomial tree of order k

4
=6
2

depth 0
depth 1

depth 2
depth 3
depth 4

45

B4

30

23

32

24

22

48

29

10

31

17

44

37

18

50

55

B4
23

B1

B0
24

Binomial Heap: Implementation

Binomial Heap: Properties

Implementation.

Properties of N-node binomial heap.

Represent trees using left-child, right sibling pointers.

three links per node (parent, left, right)

Roots of trees connected with singly linked list.

degrees of trees strictly decreasing from left to right

heap

29

10

44

37

18

17

Contains binomial tree Bi iff bi = 1 where bn b2b1b0 is binary

representation of N.
At most log2 N + 1 binomial trees.
Height log2 N.

18

37

48

50

10

31

17

44

37

30

23

32

24

22

48

31

18

N = 19
# trees = 3
height = 4
binary = 10011

10

45
50

29

t
gh
Ri

31

29

ft
Le
48

nt
re
a
P

Min key contained in root of B0, B1, . . . , Bk.

17

50

44

55

Binomial Heap

Leftist Power-of-2 Heap

B4

B1

B0

25

26

Binomial Heap: Union

Binomial Heap: Union

Create heap H that is union of heaps H and H.

"Mergeable heaps."
Easy if H and H are each order k binomial trees.
connect roots of H and H
choose smaller key to be root of H

30
45

32

23

22

24

48

29

10

31

17

45

44

30

23

32

24

22

48

29

10

31

17

44

37

28

55

33

25

12

41

19 + 7 = 26
H

50

50

55

15

18

H
27

28

Binomial Heap: Union

Binomial Heap: Union

12

18

30

23

22

48

29

10

31

17

44

37

15
45

32

24

18

45
28

33

23

32

24

22

48

10

31

17

44

37

12

50

55

30

29

33

25

12

50

25

41

15

18

28

55

41

29

12

37

18

30

25

28

15

33

25

37

12

37

18

25

41

30

23

22

48

29

10

31

17

44

37

15
45

55

32

24

18

45
33

23

32

24

22

48

10

31

17

44

37

12

50
28

30

29

25

41

55

15

33

25

12

50
28

41

12

12

18

18

31

18

32

28

15

33

25

37

12

37

18

25

28

41

30

23

22

48

29

10

31

17

32

24

44

37

18

45
28

18

15

33

25

30

23

32

24

22

48

29

10

31

17

44

37

12

37

18

15

33

25

18

12

50
28

41

45

30

23

32

24

22

48

29

10

31

17

44

28

50

15

33

25

12

37

18

41

55

33

34

Binomial Heap: Union

Binomial Heap: Delete Min

Create heap H that is union of heaps H and H.

Delete node with minimum key in binomial heap H.

Analogous to binary addition.

Running time. O(log N)

37

25

55

41

25

25

33

41

28

33

37

12

50

55

12

41

15
45

15

Find root x with min key in root list of H, and delete

H broken binomial trees
H Union(H, H)

Proportional to number of trees in root lists 2( log2 N + 1).

Running time. O(log N)

19 + 7 = 26

45

30

23

32

24

22

48

29

10

31

17

44

37

18

50

55
35

36

Binomial Heap: Delete Min

Binomial Heap: Decrease Key

Delete node with minimum key in binomial heap H.

Decrease key of node x in binomial heap H.

Find root x with min key in root list of H, and delete

H broken binomial trees

Suppose x is in binomial tree Bk.

Bubble node x up the tree if x is too small.

H Union(H, H)
Running time. O(log N)

Running time. O(log N)

45

30

23

32

24

22

48

29

10

31

17

44

18

depth = 3

37

H
x

50

55

Proportional to depth of node x log2 N .

30

23

32

24

22

48

29

10

31

17

44

37

18

50

55
37

38

Binomial Heap: Delete

Binomial Heap: Insert

Delete node x in binomial heap H.

Insert a new node x into binomial heap H.

Decrease key of x to -.

Delete min.

Running time. O(log N)

H MakeHeap(x)
H Union(H, H)

Running time. O(log N)

45

30

23

32

24

22

48

29

10

31

17

44

37

18

50

55
39

40

Binomial Heap: Sequence of Inserts

Insert a new node x into binomial heap H.
If N = .......0, then only 1 steps.

If N = ......01, then only 2 steps.

If N = .....011, then only 3 steps.

If N = ....0111, then only 4 steps.

48

29

10

31

17

Priority Queues

44

37

Heaps

50

Inserting 1 item can take (log N) time.

If N = 11...111, then log2 N steps.

But, inserting sequence of N items takes O(N) time!

(N/2)(1) + (N/4)(2) + (N/8)(3) + . . . 2N

Amortized analysis.
Basis for getting most operations
down to constant time.

nn
n=1 2

= 2 NN N11
2
2
2

Operation

Linked List

Binary

Binomial

Fibonacci *

Relaxed

make-heap

insert

log N

log N

find-min

log N

delete-min

log N

log N

log N

log N

union

log N

decrease-key

log N

log N

delete

log N

log N

log N

log N

is-empty

just did this

41

42