Computer Science and Engineering| Indian Institute of Technology Kharagpur

cse.iitkgp.ac.in

Algorithms – I (CS21203)

Autumn 2023, IIT Kharagpur

Priority Queues and Heaps

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Resources
• Apart from the book
• UC Davis ECS 36C Course by Prof. Joël Porquet-Lupine

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 2

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Introduction
• Let us think about a scenario where you are performing the
following tasks simultaneously
• Start a long code compilation
• Refresh moodle page
• Send Gmail chat message
• Receive email notification
• Objective: Execute multiple processes until completion, but keep
the system responsive
1 2 3 4

• Are processes executed until completion?

• Will the system feel responsive?
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 3
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Process Scheduling
• Round-robin scheduling
• Add processes to a (typically FIFO) queue
• Execute them for equal chunks of time
1 2 3 4

• Will the system feel more responsive? Can we do better?

• Priority scheduling
• Associate priority with each process, e.g., short processes get higher priority
• Execute the processes with the highest priority first
1 2 3 4 A very good resource on process scheduling:
Prof. Krzyzanowski’s lecture notes

Source: UC Davis, ECS 36C course, Spring

4
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Priority Queue
• Definition: Queue where each item is associated to a priority (i.e., a
comparable key)
• Push/Insert: add item and associated priority
• Pop/Pull: remove item with highest priority

Source: UC Davis, ECS 36C course, Spring

5
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Naïve Implementations

Source: UC Davis, ECS 36C course, Spring

6
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

List Implementations
• Struct node

• Function to create a new node

• Function to check if queue is empty

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 7

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

List Implementations
• Get value of the maximum priority (minimum element)

• Extract the element with maximum priority (minimum element)

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 8

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

List Implementations
• Push new element at the right place

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 9

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

List Implementations
• Complexities?
• getMin() -
• pop() -
• push() -

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 10

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Binary Search Tree Implementations

• A priority queue can also be entirely mapped onto a BST
implementation
• getMin()
• Use getMin() -
•pop()
• Use delete(getMin()) -
•push()
• Use insert() method of BST -

•Conclusion on naïve implementations

• A list implementation gives getMin()/pop() in
• But push() in
• A BST implementation gives push() in
• But getMin()/pop() in as well
• How to get the best of both worlds?
Source: UC Davis, ECS 36C course, Spring
11
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Binary Heap
• Most common data structure for implementing a priority queue
• So ubiquitous in implementing priority queues that the word ‘heap’
is used without any qualifier in this context
• A heap is a binary tree with two properties
• Heap-order property
• Structure property
• Operations on heaps can destroy one or more of these properties
• So a heap operation must not terminate until all heap properties
are in order
2
18 42

51 99 93
Source: UC Davis, ECS 36C course, Spring
12
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heap-Order Property
• Since we want to find the minimum quickly, it makes sense that the
smallest element should be at the root.
• Continuing, any node should be smaller than all of its descendants
• Giving two types of Heaps
• min-heaps: each node's key is less than or equal to each of its children
• max-heaps: each node's key is greater than or equal to each of its children

2 2
18 42 51 42
Example of min heap Example of broken heap

51 99 93 18 99 93
•By design, item with highest priority is always the root
• getMin() is
Source: UC Davis, ECS 36C course, Spring
13
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Structure Property A
•A heap is a complete binary tree
• All levels are completely filled, apart B C
from possibly the last
• The last level is packed to the left D E F G
•Guarantee of height in
H I J
•Easy representation
• Complete binary tree is so regular, it can be represented in an array
• No need for complicated link management, and fast traversal
For node at index *:
•Left child at index
•Right child at index
• Root is always at index 1 •Parent (if not root) at index

*Heap could start at array index 0, but node indexing would become a little more complex
Source: UC Davis, ECS 36C course, Spring
14
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Implementation
•Min-heap version
•Simple array (capacity incremented by 1 for the unused first element)

Source: UC Davis, ECS 36C course, Spring

15
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Implementation
• getMin() – Returns the node with highest priority (minimum value in
min-heap)
•It is as simple as returning the root only
A
B C

D E F

•Next, we will insert or push() an element into the heap

• What is the most natural position for the new element? (Don’t worry
about the heap order property, just keep the structure property intact)
Source: UC Davis, ECS 36C course, Spring
16
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Implementation
• push() – Insert only at locations that • If heap-order not broken, stop!
keeps the tree complete • Otherwise, swap with parent
2 2
18 54 18 54

42 43 99 93 15 43 99 93

52 15 52 42
• Continue going up tree until • Strategy known as shift up
heap-order is respected • Also called bubble up, heapify-
2 up
15 54

18 43 99 93

52 42
Source: UC Davis, ECS 36C course, Spring
17
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Implementation
• Continue going up tree until • Strategy known as shift up
heap-order is respected • Also called bubble up, heapify-
2 up
15 54

18 43 99 93

52 42
• Insertion of new item

Source: UC Davis, ECS 36C course, Spring

18
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Implementation
• pop() – Removing min item leaves • If heap-order not broken, stop!
hole at root. Move last item to root • Otherwise, swap with smallest child
to keep the2 tree complete
93 18
18 54 93 54

42 43 99 93 42 43 99
• Continue going down tree until • Strategy known as shift down
heap-order is respected again • Also called bubble down, heapify-
down
18
42 54

93 43 99

Source: UC Davis, ECS 36C course, Spring

19
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Implementation
• Strategy known as shift down
• Also called bubble down, heapify-
down

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 20

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Conclusion
• Running time complexities

• Other operations:
• In-place item modification, e.g., when heap/priority queue is
used for process scheduling, change priority of a process
• DecreaseItem(Item, DeltaPriority)
• IncreaseItem(Item, DeltaPriority)
•Heap variants:
• 2-3 heap, Binomial heaps, d-ary heaps, Fibonacci heaps,
Leftist heaps, Skew heaps etc.

Source: UC Davis, ECS 36C course, Spring

21
2020
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap
• Sometimes, build an entire heap directly out of an initial collection
of items
• One use-case is Heapsort – an efficient way to sort an array
• Naïve approach (also known as Williams' method)
• For all items, push() times
• Each push() operation takes average time and worst-case time
• For items, this algorithm will run in average time and worst-case
time
• Is there a better way?

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 22

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap
• Linear approach (Floyd's method)
• Put all the elements on binary tree, only respecting structure property
• Shift down all the elements starting from the first node who has at least
one child, and up to the root

51 51
43 93 43 93

18 42 99 54 2 42 99 54

2 74 18 74

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 23

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap
51 51
43 54 2 54

2 42 99 93 18 42 99 93

18 74 43 74

2
18 54

43 42 99 93

51 74

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 24

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap Implementations

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 25

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap Complexity Analysis

• The running time of heapify operation is where is the number of
elements in the heap
• RECAP - Height: The height of a node is the length of the longest
path from a leaf to this node
A
B C

D E F G H

I J K
• All leaves are at height 0
• The height of a tree is equal to the height of the root from the
deepest
Oct 04, 06 2023
leaf (which is always equal to the depth of the tree)
CS21203 / Algorithms - I | Priority Queue, Heap 26
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap Complexity Analysis

• The running time of heapify operation is where is the number of
elements in the heap
• Some intuitions first [Great resource on this topic -
https://stackoverflow.com/a/18742428]
• The basic idea is after creating a complete binary tree - move an
offending node until it satisfies the heap property
• shiftUp - swaps a node that is less than its parent (thereby moving it up)
until it is no smaller than the node above it
• shiftDown - swaps a node that is more than its smallest child (thereby
moving it down) until it is no larger than both nodes below it
• The number of operations required for shiftUp and shiftDown is
proportional to the distance the node may have to move
• For shiftUp, it is the distance to the top of the tree, so shiftUp is
expensive for nodes at the bottom of the tree
• For shiftDown, it is the distance to the bottom of the tree, so
shiftDown is expensive for nodes at the top of the tree
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 27
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap Complexity Analysis

• Although both operations are O(log n) in the worst case, only one
node is at the top whereas half the nodes lie in the bottom layer
• So it shouldn't be too surprising that if we have to apply an
operation to every node, we would prefer shiftDown over shiftUp

Max # of Max # of
swaps nodes
A 0
height 1= 2
1
B C height 2=2
D E F G height 4=2
2

I J K L M N O P 3
height 8=2
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 28
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap Complexity Analysis

• Total swaps (max) =

Max # of Max # of
swaps nodes
A 0
height 1= 2
1
B C height 2=2
D E F G height 4=2
2

I J K L M N O P 3
height 8=2
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 29
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapify/BuildHeap Complexity Analysis

• Total swaps (max) =

• (2) – (1)

Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 30

 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapsort
• Heapsort uses heaps to sort in time
• Basic strategy:
• Build a min-heap of elements in time
• Perform pop() operations
• Store these elements in a second array giving sorted elements
• Since each pop() takes times, total running time is
• The main problem with this algorithm is that it uses an extra array
• A clever way to avoid this is to use the fact that after each pop()
operation, the heap shrinks by 1
• Thus, the previous last cell can be used to store the element that was
just popped
• Using this strategy, the array will contain the elements in decreasing
sorted order
• If we want in more typical increasing sorted order, we can change the
heap to max-heap so that parent has larger element than child
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 31
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapsort
1 1
12 9
2 3 2 3
6 10 6 10
5 6
Swap 5 6
4 4
5 1 9 5 1 12
0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 12 6 10 5 1 9 0 9 6 10 5 1 12

Remove
1 1
10 9
2 3 2 3
6 9 6 10
ShiftDown
4 5 4 5 6
5 1 5 1 12
0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 10 6 9 5 1 12 0 9 6 10 5 1 12
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 32
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapsort
1 1
10 1
2 3 2 3
6 9 6 9
5 Swap 5
4 4
5 1 5 10
0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 10 6 9 5 1 12 0 1 6 9 5 10 12

Remove
1 1
9 1
2 3 2 3
6 1 6 9
ShiftDown
4 4 5
5 5 10
0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 9 6 1 5 10 12 0 1 6 9 5 10 12
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 33
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapsort
1 1
9 5
2 3 2 3
6 1 6 1
Swap
4 4
5 9
0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 9 6 1 5 10 12 0 5 6 1 9 10 12

Remove
1 1
6 5
2 3 2 3
5 1 6 1
ShiftDown
4
9
0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 6 5 1 9 10 12 0 5 6 1 9 10 12
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 34
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapsort
1 1
6 1
2 3 2 3
5 1 5 6
Swap

0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 6 5 1 9 10 12 0 1 5 6 9 10 12

Remove
1 1
5 1
2 2 3
1 5 6
ShiftDown

0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 5 1 6 9 10 12 0 1 5 6 9 10 12
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 35
 Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Heapsort
1 1
5 1
2 2
1 5
Swap

0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 5 1 6 9 10 12 0 1 5 6 9 10 12

Remove
1 1
1 1
2
5

0 1 2 3 4 5 6 0 1 2 3 4 5 6
0 1 5 6 9 10 12 0 1 5 6 9 10 12
Oct 04, 06 2023 CS21203 / Algorithms - I | Priority Queue, Heap 36
Computer Science and Engineering| Indian Institute of Technology Kharagpur
cse.iitkgp.ac.in

Thank You