Chap 2: Heap and Priority Q 01-10-2020

Chapter 2

Syllabus Topics
• Heap Operations
• Implementation of Priority Queue and Heap

Neepa Shah

Neepa Shah 1
Chap 2: Heap and Priority Q 01-10-2020

Priority Queue ADT

• A priority queue stores a collection of entries

• Each entry is a pair (key, value)

• Main methods of the Priority Queue ADT

▫ insert(k, x)
▫ removeMin()

• Additional methods
▫ min()
▫ int size()
▫ int isEmpty()

Neepa Shah

Applications of Priority Queues

• Auctions

• Any event/job management that assign priority

to events/jobs

• In Operating Systems
▫ Scheduling jobs

• In Simulators
▫ Scheduling the next event (smallest event time)
Neepa Shah

Neepa Shah 2
Chap 2: Heap and Priority Q 01-10-2020

Example
Operation Output Priority Queue
insert(5, A) [= (5, A)] {(5, A)}
Insert(9, C) [= (9, C)] {(5, A), (9, C)}
Insert(3, B) [= (3, B)] {(3, B), (5, A), (9, C)}
Insert(7, D) [= (7, D)] {(3, B), (5, A), (7, D), (9, C)}
min() e3 {(3, B), (5, A), (7, D), (9, C)}
removeMin() e3 {(5, A), (7, D), (9, C)}
size() 3 {(5, A), (7, D), (9, C)}
removeMin() e1 {(7, D), (9, C)}
removeMin() e4 {(9, C)}
removeMin() e2 {}
removeMin() “error” {}
isEmpty() true {}
Neepa Shah

Priority Queue Sorting

• We can use a priority queue to sort a set of comparable
elements
1. Insert the elements one by one with a series of
insert operations
2. Remove the elements in sorted order with a series
of removeMin operations

• The running time of this sorting method depends on

the priority queue implementation

Neepa Shah

Neepa Shah 3
Chap 2: Heap and Priority Q 01-10-2020

Implementing a Priority Queue with a list

• Implementation with an unsorted list
▫ Fast insertions and slow removals

• Implementation with a sorted list

▫ Slow insertions and fast removals

Neepa Shah

Sequence-based Priority Queue

• Implementation • Implementation
with an unsorted with a sorted list
list
1 2 3 4 5
4 5 2 3 1

• Performance: • Performance:
▫ insert takes O(1) ▫ insert takes O(n) time
time ▫ removeMin and min
▫ removeMin and take O(1) time
min take O(n)
time

Neepa Shah

Neepa Shah 4
Chap 2: Heap and Priority Q 01-10-2020

Comparison

Method Unsorted list Sorted list

Size, isEmpty O(1) O(1)
insert O(1) O(n)
min, removeMin O(n) O(1)

Neepa Shah

10

Insertion-Sort Neepa Shah

Neepa Shah 5
Chap 2: Heap and Priority Q 01-10-2020

11

Insertion-Sort Contd.
• Insertion-sort is the variation of PQ-sort where
the priority queue is implemented with a sorted
sequence
• Running time of Insertion-sort:
1. Inserting the elements into the priority queue with n
insert operations takes time proportional to
1 + 2 + …+ n
2. Removing the elements in sorted order from the
priority queue with a series of n removeMin operations
takes O(n) time
• Insertion-sort runs in O(n2) time

Neepa Shah

12

Insertion-Sort Example
Sequence S Priority queue P
Input: (7,4,8,2,5,3,9) ()

Phase 1
(a) (4,8,2,5,3,9) (7)
(b) (8,2,5,3,9) (4,7)
(c) (2,5,3,9) (4,7,8)
(d) (5,3,9) (2,4,7,8)
(e) (3,9) (2,4,5,7,8)
(f) (9) (2,3,4,5,7,8)
(g) () (2,3,4,5,7,8,9)

Phase 2
(a) (2) (3,4,5,7,8,9)
(b) (2,3) (4,5,7,8,9)
.. .. ..
. . .
(g) (2,3,4,5,7,8,9) ()
Neepa Shah

Neepa Shah 6
Chap 2: Heap and Priority Q 01-10-2020

13

Insertion-Sort Example Contd.

• Illustrate the performance of insertion-sort on
the following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)

Neepa Shah

Selection Sort - Example

32 91 12 55 74 73 18

Neepa Shah 7
Chap 2: Heap and Priority Q 01-10-2020

Selection Sort – Example Contd.

SWAP !

32 91 12 55 74 73 18

16

Selection-Sort
• Selection-sort is the variation of PQ-sort where the
priority queue is implemented with an unsorted
sequence
• Running time of Selection-sort:
1. Inserting the elements into the priority queue with n insert
operations takes O(n) time
2. Removing the elements in sorted order from the priority
queue with n removeMin operations takes time proportional
to
n + n-1 + … + 2 + 1
• Selection-sort runs in O(n2) time
Neepa Shah

Neepa Shah 8
Chap 2: Heap and Priority Q 01-10-2020

17

Selection-Sort Example
Sequence S Priority Queue P
Input: (7,4,8,2,5,3,9) ()

Phase 1
(a) (4,8,2,5,3,9) (7)
(b) (8,2,5,3,9) (7,4)
.. .. ..
. . .
(g) () (7,4,8,2,5,3,9)

Phase 2
(a) (2) (7,4,8,5,3,9)
(b) (2,3) (7,4,8,5,9)
(c) (2,3,4) (7,8,5,9)
(d) (2,3,4,5) (7,8,9)
(e) (2,3,4,5,7) (8,9)
(f) (2,3,4,5,7,8) (9)
(g) (2,3,4,5,7,8,9) ()
Neepa Shah

18

Selection-Sort Example Contd.

• Illustrate the performance of selection-sort on
the following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)

Neepa Shah

Neepa Shah 9
Chap 2: Heap and Priority Q 01-10-2020

Heaps

5 6

9 7

20

Why study Heap?

• Heap is always O(n log n)

Neepa Shah

Neepa Shah 10
Chap 2: Heap and Priority Q 01-10-2020

21

Neepa Shah

The Priority Queue using HEAP

• DeleteMin
▫ log N time
• Insert
▫ log N time
• Other operations
▫ FindMin, Min, isEmpty
 Constant time
▫ Initialize
 N time

21

22

Heaps and Priority Queues Contd.

• We can use a heap to implement a priority queue
• We store a (key, element) item at each internal node
• We keep track of the position of the last node

(2, ABC)

(5, XYZ) (6, PQ)

(9, BCD) (7, QR)

Neepa Shah

Neepa Shah 11
Chap 2: Heap and Priority Q 01-10-2020

23

What is a “heap”?
• Definition of heap:
A balanced, left-justified binary tree in
which no node has a value greater than /
lesser than the value in its parent

Neepa Shah

24

Balanced binary trees

• The depth of a node is its distance from the root
• The depth of a tree is the depth of the deepest node

• A binary tree of depth n is balanced if all the nodes

at depths 0 through n-2 have two children

n-2
n-1
n
Balanced Balanced Not balanced

Neepa Shah

Neepa Shah 12
Chap 2: Heap and Priority Q 01-10-2020

25

Left-justified binary trees

• A balanced binary tree of depth n is left-
justified if:
▫ it has 2n nodes at depth n (the tree is “full”)
or
▫ it has 2k nodes at depth k, for all k < n, and all
the leaves at depth n are as far left as possible

Left-justified Not left-justified

Neepa Shah

26

Neepa Shah

The (MAX) heap property

• A node has the heap property if the value in the node is larger
than the values in its children

12 12 12

8 3 8 12 8 14
Blue node has heap Blue node has heap Blue node does not have
property property heap property

• All leaf nodes automatically have the heap property

• A binary tree is a heap if all nodes in it have the heap property

Neepa Shah 13
Chap 2: Heap and Priority Q 01-10-2020

27

Neepa Shah

The (MIN) heap property

• A node has the heap property if the value in the node is smaller
than the values in its children

3 1 12

8 13 4 12 8 14
Blue node has heap Blue node has heap Blue node does not have
property property heap property

• All leaf nodes automatically have the heap property

• A binary tree is a heap if all nodes in it have the heap property

28

Neepa Shah

Examples

Neepa Shah 14
Chap 2: Heap and Priority Q 01-10-2020

29

Height of a Heap
• Theorem: A heap storing n keys has height O(log n)
Proof: (we apply the complete binary tree property)
▫ Let h be the height of a heap storing n keys
▫ Since there are 2i keys at depth i = 0, … , h - 1 and at least one key at
depth h, we have n  1 + 2 + 4 + … + 2h-1 + 1
▫ Thus, n  2h , i.e., h  log n

depth keys
0 1

1 2

h-1 2h-1

h 1

Neepa Shah

30

Neepa Shah

siftUp (MAX / MIN HEAP)

• Given a node that does not have the heap property,
you can give it the heap property by exchanging its
value with the value of the larger /smaller child

12 14

8 14 8 12
Blue node does not have Blue node has heap
heap property property

• This is sometimes called sifting up

Neepa Shah 15
Chap 2: Heap and Priority Q 01-10-2020

31

Constructing a heap I
• A tree consisting of a single node is
automatically a heap
• We construct a heap by adding nodes one at a
time:
▫ Add the node just to the right of the rightmost
node in the deepest level
▫ If the deepest level is full, start a new level
• Examples:
Add a new Add a new
node here node here

Neepa Shah

32

Constructing a heap Contd.

• Each time we add a node, we may destroy the
heap property of its parent node
• To fix this, we sift up

Neepa Shah

Neepa Shah 16
Chap 2: Heap and Priority Q 01-10-2020

33

Neepa Shah

Constructing a heap Contd.

8 8 10 10

10 8 8 5

1 2 3

10 10 12

8 5 12 5 10 5

12 8 8
4

34

Heap Operations
• insert (k, x)
• deleteMin()

Neepa Shah

Neepa Shah 17
Chap 2: Heap and Priority Q 01-10-2020

35

Insertion Example: insert(14): MIN HEAP

Neepa Shah

14

14

14

36

Basic Heap Operations: insert(x)

• Maintain the complete binary tree property and
fix any problem with the partially ordered tree
property
▫ Create a leaf at the end
▫ Repeat
 Locate parent
 if POT (Partially Ordered Tree) not satisfied
 Swap with parent
 else
 Stop
▫ Insert x into its final location
Neepa Shah

Neepa Shah 18
Chap 2: Heap and Priority Q 01-10-2020

37

Neepa Shah

• Here’s a sample binary tree after it has been heapified

A sample heap (MAX)

25

22 17

19 22 14 15

18 14 21 3 9 11

• Notice that heapified does not mean sorted

• Heapifying does not change the shape of the binary
tree; this binary tree is balanced and left-justified

38

Removing the root

Neepa Shah

• Notice that the largest number is now in the root

• Suppose we discard the root:
11

22 17

19 22 14 15

18 14 21 3 9 11

• How can we fix the binary tree so it is once again

balanced and left-justified?
• Solution: remove the rightmost leaf at the deepest
level and use it for the new root

Neepa Shah 19
Chap 2: Heap and Priority Q 01-10-2020

Neepa Shah 39

The reHeap method Contd.

• Our tree is once again a heap, because every node in
it has the heap property
22

22 17

19 21 14 15

18 14 11 3 9

• We can repeat this process until the tree becomes empty

Since a heap has height O(log n), downheap runs in O(log n) time

Mapping into an array

25

22 17

19 22 14 15

18 14 21 3 9 11

0 1 2 3 4 5 6 7 8 9 10 11 12
25 22 17 19 22 14 15 18 14 21 3 9 11
• Note:
▫ The left child of index i is at index 2*i+1
▫ The right child of index i is at index 2*i+2
Neepa Shah
40

Neepa Shah 20
Chap 2: Heap and Priority Q 01-10-2020

Neepa Shah 41

Removing and replacing the root

• The “root” is the first element in the array
• The “rightmost node at the deepest level” is the last
element
• Swap them...

0 1 2 3 4 5 6 7 8 9 10 11 12
25 22 17 19 22 14 15 18 14 21 3 9 11

0 1 2 3 4 5 6 7 8 9 10 11 12
11 22 17 19 22 14 15 18 14 21 3 9 25

• ...And pretend that the last element in the array no longer

exists—that is, the “last index” is 11 (containing the value
9)

Reheap and repeat

Neepa Shah 42

• Reheap the root node (index 0, containing 11)...

0 1 2 3 4 5 6 7 8 9 10 11 12
11 22 17 19 22 14 15 18 14 21 3 9 25

0 1 2 3 4 5 6 7 8 9 10 11 12
22 22 17 19 21 14 15 18 14 11 3 9 25

0 1 2 3 4 5 6 7 8 9 10 11 12
9 22 17 19 22 14 15 18 14 21 3 22 25

 ...And again, remove and replace the root node

• Repeat until the last becomes first, and the array is

sorted!

Neepa Shah 21
Chap 2: Heap and Priority Q 01-10-2020

43

Heap Operations
• insert (k, x)
• deleteMin()

Neepa Shah

44

Neepa Shah

deleteMin() example: MIN HEAP

31

13 14 16 19 21 19 68 65 26 32 31

Neepa Shah 22
Chap 2: Heap and Priority Q 01-10-2020

45

Neepa Shah

deleteMin() Example (Cont’d)

31

31

31

46

Basic Heap Operations: deleteMin()

• Replace root with the last leaf (last element in
the array representation
▫ This maintains the complete binary tree property
but may violate the partially ordered tree property
• Repeat
▫ Find the smaller child of the “new root”
▫ If POT not satisfied
 Swap “new root” and smaller child
▫ else
 Stop

Neepa Shah

Neepa Shah 23
Chap 2: Heap and Priority Q 01-10-2020

47

Example Neepa Shah

heapify(6)

heapify(5) heapify(4)

48

Neepa Shah

heapify(3) heapify(2)

heapify(1) heapify(0)
Since a heap has height O(log n), heapify runs in O(log n) time

Neepa Shah 24
Chap 2: Heap and Priority Q 01-10-2020

Sorting
• What do heaps have to do with sorting an array?
▫ Because the binary tree is balanced and left justified,
it can be represented as an array
▫ All our operations on binary trees can be represented
as operations on arrays
▫ To sort:
heapify the array;
while the array isn’t empty {
remove and replace the root;
reheap the new root node;
}

Neepa Shah
49

Analysis I
• Here’s how the algorithm starts:
heapify the array;

• Heapifying the array: we add each of n nodes

▫ Each node has to be sifted up, possibly as far as
the root
 Since the binary tree is perfectly balanced, sifting
up a single node takes O(log n) time
▫ Since we do this n times, heapifying takes
n*O(log n) time, that is, O(n log n) time

Neepa Shah
50

Neepa Shah 25
Chap 2: Heap and Priority Q 01-10-2020

Analysis II
• Rest of the algorithm:
while the array isn’t empty {
remove and replace the root;
reheap the new root node;
}
• We do the while loop n times (actually, n-1
times), because we remove one of the n nodes
each time
• Removing and replacing the root takes O(1)
time
• Therefore, the total time is n times however
long it takes the reheap method
Neepa Shah
51

Analysis III
• To reheap the root node, we have to follow one
path from the root to a leaf node (and we might
stop before we reach a leaf)
• The binary tree is perfectly balanced
• Therefore, this path is O(log n) long
▫ And we only do O(1) operations at each node
▫ Therefore, reheaping takes O(log n) times
• Since we reheap inside a while loop that we do n
times, the total time for the while loop is
n*O(log n), or O(n log n)
Neepa Shah
52

Neepa Shah 26
Chap 2: Heap and Priority Q 01-10-2020

Analysis IV
• Here’s the algorithm again:
heapify the array;
while the array isn’t empty {
remove and replace the root;
reheap the new root node;
}
• Heapifying takes O(n log n) time
• The while loop takes O(n log n) time
• The total time is therefore O(n log n) + O(n log
n)
• This is the same as O(n log n) time
Neepa Shah
53

54

Heap-Sort: Conclusion
• A priority queue with n items implemented by means
of a heap
▫ The space used is O(n)
▫ Methods insert and removeMin take O(log n) time
▫ Methods size, isEmpty, and min take time O(1) time

• Using a heap-based priority queue, we can sort a

sequence of n elements in O(n log n) time

Neepa Shah

Neepa Shah 27
Chap 2: Heap and Priority Q 01-10-2020

55

Heap Tree Construction

• Two Approaches
▫ Elements: 40, 80, 35, 90, 45, 50, and 70

▫ Elements: 100, 119, 118, 171, 112, 151 and 132

Neepa Shah

56

Example

Neepa Shah

Neepa Shah 28
Chap 2: Heap and Priority Q 01-10-2020

57

Algorithm Insert

Neepa Shah

58

Delete Maximum

Neepa Shah

Neepa Shah 29
Chap 2: Heap and Priority Q 01-10-2020

59

Algorithm Adjust

Neepa Shah

60

Heap Sort

Neepa Shah

Neepa Shah 30
Chap 2: Heap and Priority Q 01-10-2020

61

Basic procedures on Heap

• Insert
• Del-Max / Extract-Min
• Adjust
• Heap Sort

• Build-Heap / Heapify

Neepa Shah

62

Heap Tree Construction

• Two Approaches
▫ Elements: 40, 80, 35, 90, 45, 50, and 70->
WORST AND AVERAGE CASE

Neepa Shah

Neepa Shah 31
Chap 2: Heap and Priority Q 01-10-2020

63

Heap Tree Construction Contd.

• Two Approaches: Procedure ADJUST
• An algorithm which regards A(l:n) as a complete
binary tree and works from the leaves up to the
root, level by level. At each level, it will be the
case that the left and right subtrees of any node
are heaps. Only the value in the root node may
violate the heap property of any node are heaps.
• Given n elements in A(l:n) we can create a heap
by applying ADJUST.

Neepa Shah

64

Heap Tree Construction Contd.

Elements: 100, 119, 118, 171, 112, 151 and 132

Neepa Shah

Neepa Shah 32
Chap 2: Heap and Priority Q 01-10-2020

65

Heap Tree Construction Contd.

Neepa Shah

66

Heap Sort: Approach 2

Neepa Shah

Neepa Shah 33