CS213/293 Data Structure and Algorithms 2023

Lecture 12: Heap

Instructor: Ashutosh Gupta

IITB India

Compile date: 2023-09-15

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 1
 Topic 12.1

Priority queue

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 2
Scheduling problem

On a computational server, users are submitting jobs to run on a single CPU.

▶ A user also declares the expected run time of the job.
▶ Jobs can be preempted.

Policy: shortest remaining processing time, which allows interruption of a job if a new job with
smaller run time is submitted.

The policy minimizes average waiting time.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 3
Scheduling problem operations

We need the following operations for the scheduling problem.

▶ Update the remaining time in every tick
▶ Delete a job when the remaining time is zero
▶ Find the next job to run
▶ Insert a job when arrives

Definition 12.1
In a priority queue, we dequeue the highest priority element from the enqueue elements with
priorities.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 4
Interface of priority queue https://en.cppreference.com/w/cpp/container/priority_queue

▶ priority_queue<T,Container,Compare> q : allocates new queue q

▶ q.push(e) : adds the given element e to the queue.
▶ q.pop() : removes the highest priority element from the queue.
▶ q.top() : access the highest priority element.

▶ Container class defines the physical data structure where the queue will be stored. The
default value is Vector .
▶ Compare class defines the method of comparing priorities of two elements.

Exercise 12.1
Give an implementation for the scheduling problem using the C++ priority queue.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 5
 Topic 12.2

Implementations of priority queue

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 6
Implementation using unsorted linked list/array

In case we use a linked list,

▶ We implement q.push by inserting the element at the front of the linked list, which is O(1)
operation.
▶ We need to scan the entire list to find the maximum for implementing q.pop and q.top

Exercise 12.2
How will we implement a priority queue over unsorted arrays?

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 7
Implementation using sorted linked list/array

In case we use a linked list,

▶ The maximum will be at the end of the list. We can implement q.pop and q.top in O(1).

▶ However, q.push(e) needs to scan the entire list to find the right place to insert e, which is
O(n) operation.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 8
 Priority queue

Priority queue is one of the fundamental containers.

Many other algorithms assume access to efficient priority queues.

We will define a data structure heap that provides an efficient implementation for the priority
queue.

Commentary: Heap is like the the red-black tree, which provides an efficient implementation for ordered maps.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 9
 Topic 12.3

Heap - somewhat sorting!

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 10
Heap

Definition 12.2 Example 12.1

A heap T is a binary tree such that the An example of heap.
following holds.
▶ (structural property) All levels are full 21
except the last one and the last level is
left filled. 20 19
▶ (heap property) for each non-root node n,
key (n) ≤ key (parent(n)). 16 13 17

Exercise 12.3
Show that nodes on a path from root to a leaf have keys in non-increasing order.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 11
Exercise: Identify Heap

Exercise 12.4
Which of the following are Heaps?

17 26 75 21

5 11 22 21 21 17 17

6 18 17 17 5 17

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 12
Algorithm: maximum

Algorithm 12.1: Maximum(Heap T)

return T [0]

▶ Correctness
▶ Let us suppose the maximum is not at the root.
▶ There is a node n that has maximum key but parent(n) has greater key, which violates heap
condition.
▶ Contradiction.

▶ Running time is O(1).

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 13
Height of heap

Let us suppose a heap has n nodes and height h.

The number of nodes in a complete binary tree of height h is 2h − 1.

Therefore,
2h−1 − 1 < n ≤ 2h − 1.

Therefore n = ⌊log2 n⌋
Exercise 12.5
Give an example of a heap that touches the lower bound.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 14
Storing heap
Let us number the nodes of a heap in the order of level.

0 20
1 17 2 19
3 8 4 11 5 14 6 4
7 1 8 12 9 10 10 0 11 5 12 9 13 3 14 2

parent(i) = (i − 1)/2, left(i) = 2i + 1, and right(i) = 2i + 2.

We place the nodes on an array and traverse the heap using the above equations.
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
20 17 19 8 11 14 4 1 12 10 0 5 9 3 2

Since the last level is left filled, we are guaranteed the nodes are contiguously placed.
Instead of writing key (i) of node i in heap T , we will write T [i] to indicate the key.
cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 15
 Topic 12.4

Insert in heap

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 16
Example: Insert in Heap
Example 12.2
Where do we insert 15?

0 21 0 21
Insert at end
Repair heap
1 13 2 17 1 15 2 17

3 10 4 7 5 11 6 2 3 10 4 13 5 11 6 2

7 5 8 8 9 6 10 15 7 5 8 8 9 6 10 7

▶ Insert at the first available place, which is easy to spot. (Why?)

▶ Move up the new key if the heap property is violated.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 17
Algorithm: Insert
▶ Correctness
▶ Structural property holds due to the
insertion position.
▶ Due to the heap property of input T , the
Algorithm 12.2: Insert(Heap T, key k)
path to i the nodes must be in
1 i := T .size; non-increasing order.
2 T [i] := k; ▶ Let i0 be the value of i when the loop exits.
3 while i > 0 and T [parent(i)] < T [i] do ▶ Insert replaces the keys of the nodes in
4 Swap(T, parent(i), i); the path from i0 to T .size with the keys of
5 i := parent(i) their parents, which implies the keys do not
decrease at the nodes.
6 T .size := T .size + 1; ▶ Therefore, no introduction of a violation.
▶ Therefore, we will have a heap at the end.

▶ Running time is O(log T .size).

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 18
 Topic 12.5

Heapify

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 19
Heapify : a basic operation on a heap

▶ Let i be a node of heap T

▶ Let us suppose the binary trees rooted at left(i) and right(i) are valid heaps.
▶ T [i] may be smaller than its children and violates the heap property.
▶ The method Heapify makes the binary tree rooted at i a heap by pushing down T [i] in the
tree.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 20
Example: Heapify
Example 12.3
The trees rooted at positions 1 and 2 are heaps. We have a violation at position 0. Heapify will fix
the problem by moving the key down.
0
9 0 17
Heapify(0)
1
17 2 13 1 10 2 13

3 10 4 7 5 11 6 2 3 9 4 7 5 11 6 2

7 5 8 8 9 6 7 5 8 8 9 6

▶ Keep moving down to the child which has the maximum key. (Why?)
cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 21
 Algorithm: Heapify

Algorithm 12.3: Heapify(Heap T, i)

c := IndexWithLargestKey(T , i, left(i), right(i)) //assume T [i] = −∞ if i ≥ T .size.
if c == i then return;
Swap(T , c, i);
Heapify(T ,c);
▶ Correctness
▶ Same as insert, but we are pushing down.

▶ Running time is O(log T .size).

Commentary: Assumption T [i] = −∞ if i ≥ T .size is a convenience of notation. We may have a situation, where the T [i] exists and has some non-negative infinity key.
Without loss of correctness, we can interpret the as if the key is −∞. We will need this interpretation later for HeapSort.
cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 22
 Topic 12.6

Delete maximum in heap

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 23
Example: DeleteMax

Example 12.4
Let us delete 21 at position 0.
0
21 0 17
Swap(T,0,9)
1 Heapify(0)
17 2 13 1 10 2 13

3 10 4 7 5 11 6 2 3 8 4 7 5 11 6 2

7 5 8 8 9 6 7 5 8 6

▶ Swap with the last position, delete the last position, and run Heapify.
cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 24
Algorithm: DeleteMax

Algorithm 12.4: DeleteMax(Heap T) ▶ Correctness

1 Swap(T , 0, T .size − 1); ▶ The maximum element is removed and
2 T .size := T .size − 1; heapify returns a heap.
3 Heapify(T , 0);
4 return T [T .size]; ▶ Running time is O(log T .size).

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 25
 Topic 12.7

Build heap

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 26
Build heap https://en.cppreference.com/w/cpp/algorithm/make_heap

▶ Input: A binary tree T that has the structural property

▶ If structural property holds, then the T is an array

▶ Output: A heap over elements of T

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 27
Algorithm: BuildHeap

Order of processing in BuildHeap.

Algorithm 12.5: BuildHeap(Heap T)

1 for i := T .size − 1 downto 0 do
2 Heapify(T , i)

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 28
Example: BuildHeap
Example 12.5
Consider sequence 1 17 19 20 11 9 4 8 12 10 0 5 14 3 2. Let us fill them in the following tree.

0 1

1 17 2 19

3 20 4 11 5 9 6 4

7 8 8 12 9 10 10 0 11 5 12 14 13 3 14 2

BuildHeap traverses the tree bottom up. Heapify calls apply the following swap operations.
▶ Heapify(T,5): Swap(T,5,12)
▶ Heapify(T,1): Swap(T,1,3)
▶ Heapify(T,0): Swap(T,0,1); Swap(T,1,3); Swap(T,3,8);
cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 29
Correctness of BuildHeap

▶ Correctness by induction
▶ Base case:
If i does not have children, it is already a heap.
▶ Induction step:
We know left(i) > i or right(i) > i.
Due to the induction hypothesis, both the subtrees are heap before processing i.
Therefore, Heapify(T , i) will return a heap rooted at i.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 30
Running time of BuildHeap
Heapify for i has O(height(i)) swaps.

Let us suppose T is a complete tree with n nodes.

At height h the number of nodes is ⌈n/2h+1 ⌉ and the height of T is ⌊log n⌋.

The total running time of BuildHeap is

⌊log n⌋ ⌊log n⌋
X
h+1 n X h
O(h)⌈n/2 ⌉ = O( )
2 2h
h=0

P∞ h
Since h=0 = 2, the running time is O(n).
2h
cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 31
Some calculation
We know
∞
X 1
xi =
1−x
i=0

After differentiating over x,

∞
X 1
ix i−1 =
(1 − x)2
i=0

After multiplying with x,

∞
X x
ix i =
(1 − x)2
i=0

After putting x = 1/2,

∞
X i
=2
2i
i=0

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 32
 Topic 12.8

Heapsort

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 33
 Heapsort
▶ Since DeleteMax moves maximum to
Algorithm 12.6: HeapSort(Tree T) T .size − 1 position, the array is sorted in place.
1 T .size = |nodes of T|;
▶ Running time:
2 BuildHeap(T );
▶ BuildHeap is O(n)
3 while T .size > 0 do ▶ DeleteMax(T ) is O(log i) at size i.
4 DeleteMax(T )
▶ Total running time: O(n log n).

Exercise 12.6
Both BuildHeap and the above loop have iterative runs of Heapify in them.
Why are their running time complexities different?
Commentary: Please solve the above exercise to clearly understand the relevant mathematics.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 34
Example: Heapsort
Consider the following Heap obtained after running BuildHeap.
0 20
1 17 2 19
3 8 4 11 5 14 6 4
7 1 8 12 9 10 10 0 11 5 12 9 13 3 14 2

After the first DeleteMax,

0 19
1 17 2 14
3 8 4 11 5 9 6 4
7 1 8 12 9 10 10 0 11 5 12 2 13 3 14 20
cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 35
Example: Heapsort(2)

After the second DeleteMax,

0 17
1 11 2 14
3 8 4 10 5 9 6 4
7 1 8 12 9 3 10 0 11 5 12 2 13 19 14 20

DeleateMax has placed 19 and 20 at their sorted position.

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 36
 Topic 12.9

Tutorial problems

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 37
Exercise: Why heap?

Exercise 12.7
Can a Priority Queue be implemented as a red-black tree? What advantages does a heap
implementation has over a red-black tree implementation?

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 38
Exercise: 2D-matrix

Exercise 12.8
Suppose we have a 2D array where we maintain the following conditions: for every (i,j), we have
A(i, j) ≤ A(i + 1, j) and A(i, j) ≤ A(i, j + 1). Can this be used to implement a priority queue?

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 39
 Topic 12.10

Problems

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 40
 End of Lecture 12

cbna CS213/293 Data Structure and Algorithms 2023 Instructor: Ashutosh Gupta IITB India 41