Heaps

Heap Data Structure

• It is a nearly complete binary tree.

– All levels are full, except possibly the last one,

which is filled from left to right.

– Due to this, they are commonly stored using

arrays as there is no memory wastage.

• Values in the nodes satisfy a heap property, the

specifics of which depend on the kind of heap.

6
Array Representation of Heaps
• A heap can be stored as an array A.
– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]
– Number of elements in the array = A.length
– Number of elements in the heap which are stored within
array A = A.heap-size
– 0 ≤ A.heap-size ≤ A.length
• The elements in the subarray A[(n/2+1) .. n] are
leaves 7
Contd…

8
Types of Heaps
• Max-heaps (largest element at root), satisfy the
1
max-heap property: 9
– for every node i other than the root, 2 3
5 8
A[PARENT(i)] ≥ A[i] 4
3

• Min-heaps (smallest element at root), satisfy the

1
min-heap property: 1
– for every node i other than the root, 2 3
7 8
A[PARENT(i)] ≤ A[i] 4
9 9
Operations on Heaps
• Maintain/Restore the max-heap property
– MAX-HEAPIFY
• Create a max-heap from an unordered array
– BUILD-MAX-HEAP
• Sort an array in place
– HEAPSORT • Priority queues
– MAX-HEAP-INSERT,
– HEAP-EXTRACT-MAX,
– HEAP-INCREASE-KEY, and
– HEAP-MAXIMUM
11
 Maintaining the Heap Property
• Suppose a node is smaller than
a child
– Left and Right subtrees of i are
max-heaps
• To eliminate the violation:
– Exchange with larger child
– Move down the tree
– Continue until node is not smaller
than children

12
Maintaining the Heap Property
MAX-HEAPIFY(A, i) Assumptions:
• Left and Right subtrees of i are max-heaps
1. l = LEFT(i)
• A[i] may be smaller than its children
2. r = RIGHT(i)
3. if l ≤ A.heap-size and A[l] > A[i]
4. then largest = l
5. else largest = i
6. if r ≤ A.heap-size and A[r] > A[largest]
7. then largest = r
8. if largest  i
9. then exchange A[i] ↔ A[largest]
10. MAX-HEAPIFY(A, largest) 13
 A.heap-size = 10
Example A.length = 10
MAX-HEAPIFY(A, 2)

A[2]  A[4]

A[2] violates the heap property A[4] violates the heap property

A[4]  A[9]

Heap property restored 14

MAX-HEAPIFY Running Time
• It traces path from root to a leaf.
• In worst case length of path is h.
• Running time of MAX-HEAPIFY is O(h) or O(lg n)
– Since the height of the heap is lg n.

15
Building a Heap
• Convert an array A[1 … n], where n = A.length into a
max-heap.
• The elements in the subarray A[(n/2+1) .. n] are
leaves.
• Apply MAX-HEAPIFY on elements between 1 and n/2.
1

4
BUILD-MAX-HEAP(A) 2 3

1. A.heap-size = A.length 4
1
5 6
3
7

2 16 9 10
2. for i = A.length/2 downto 1 8 9 10

14 8 7
3. MAX-HEAPIFY(A, i) A: 4 1 3 2 16 9 10 14 8 7
19
 Example: A 4 1 3 2 16 9 10 14 8 7

i=5 i=4 i=3

1 1 1

4 4 4
2 3 2 3 2 3

1 3 1 3 1 3
4 5 6 7 4 5 6 7 4 5 6 7

8
2 9 10
16 9 10 8
2 9 10
16 9 10 8
14 9 10
16 9 10
14 8 7 14 8 7 2 8 7

i=2 i=1
1 1 1

4 4 16
2 3 2 3 2 3

1 10 16 10 14 10
4 5 6 7 4 5 6 7 4 5 6 7

8
14 9 10
16 9 3 8
14 9 10
7 9 3 8
8 9 10
7 9 3
2 8 7 2 8 1 2 4 1
20
Running Time of BUILD MAX HEAP
BUILD-MAX-HEAP(A)
1. A.heap-size = A.length
2. for i = A.length/2 downto 1 O(n)
3. MAX-HEAPIFY(A, i) O(lg n)

 Running time: O(n lg n)

• This is not an asymptotically tight upper bound.

21
Running Time of BUILD MAX HEAP
• HEAPIFY takes O(h)  the cost of HEAPIFY on a node i is
proportional to the height of the node i in the tree
h
 T (n)   ni hi   2i h  i   O(n)
h

i 0 i 0
Height Level No. of nodes
h0 = 3 (lg n) i=0 20

h1 = 2 i=1 21

h2 = 1 i=2 22

h3 = 0 i = 3 (lg n) 23

hi = h – i height of the heap rooted at level i

ni = 2i number of nodes at level i 22
Running Time of BUILD MAX HEAP
h
T (n)   ni hi Cost of HEAPIFY at level i  number of nodes at that level
i 0
h
  2i h  i  Replace the values of ni and hi computed before
i 0
h
hi h
 h i
2 Multiply by 2h both at the nominator and denominator and
i 0 2 write 2i as 1i
2
h
k
2  k
h
Change variables: k = h - i
k 0 2

k
 n k
The sum above is smaller than the sum of all elements to 
k 0 2 and h = lgn

 O(n) The sum above is smaller than 2

Running time of BUILD-MAX-HEAP: T(n) = O(n)

23
 Build Max-heap
A = [5, 3, 17, 10, 84, 19, 6, 22, 9]
1
5
2 3
3 17
4 5 6 7
10 84 19 6
8 9
1
22 9 i=4
5
A = [5, 3, 17, 10, 84, 19, 6, 22, 9]
2 3
3 17
4 5 6 7
10 84 19 6
8 9
A = [5, 3, 17, 10, 84, 19, 6, 22, 9]
22 9
24
 1
5 i=3 Contd…
2 3
3 17
4 5 6 7
22 84 19 6
8 9
10 9
A = [5, 3, 17, 22, 84, 19, 6, 10, 9] 1
5 i=2
2 3
3 19
4 5 6 7
22 84 17 6
8 9
10 9 A = [5, 3, 19, 22, 84, 17, 6, 10, 9]

25
 1
5 i=1 Contd…
2 3
84 19
4 5 6 7
22 3 17 6
8 9
10 9
1
A = [5, 84, 19, 22, 3, 17, 6, 10, 9] 84
2 3
22 19
4 5 6 7
10 3 17 6
8 9
5 9
A = [84, 22, 19, 10, 3, 17, 6, 5, 9]
26
Heapsort
• Goal:
– Sort an array using heaps.
• Idea:
– Build a max-heap from the array
– Swap the root (the maximum element) with the last
element in the array
– “Discard” this last node by decreasing the heap size
– Call MAX-HEAPIFY on the new root
– Repeat this process until only one node remains
27
Sort: 4, 7, 3, 1, 2 A=[7, 4, 3, 1, 2]

28
Sort: 4, 7, 3, 1, 2 A=[7, 4, 3, 1, 2]

MAX-HEAPIFY(A, 1) MAX-HEAPIFY(A, 1) MAX-HEAPIFY(A, 1)

MAX-HEAPIFY(A, 1)

29
HEAPSORT(A)
1. BUILD-MAX-HEAP(A) O(n)
2. for i = A.length downto 2 n – 1 times
3. do exchange A[1] with A[i] constant
4. A.heap-size = A.heap-size – 1 constant
5. MAX-HEAPIFY(A, 1) O(lg n)

• Running time: O(n lg n)

30
Priority Queues
Priority Queues
• Data structure for maintaining a set S of elements,
each with an associated value called a key.
• Max-priority queues support the following operations:
– INSERT(S, x): inserts element x into set S
– EXTRACT-MAX(S): removes and returns element of S with
the largest key
– MAXIMUM(S): returns element of S with the largest key
– INCREASE-KEY(S, x, k): increases value of element x’s key
to k (Assume k ≥ x’s current key value)

32
HEAP-MAXIMUM
Goal:
– Return the largest element of the heap

HEAP-MAXIMUM(A) Heap A:
1. return A[1]

Running time: O(1)

Heap-Maximum(A) returns 7

33
HEAP-EXTRACT-MAX
Goal: Extract the largest element of the heap (i.e.,
return the max value and also remove that
element from the heap
Idea:
– Exchange the root element with the last
– Decrease the size of the heap by 1 element
– Call MAX-HEAPIFY on the new root
Heap A: Root is the largest element

34
HEAP-EXTRACT-MAX
HEAP-EXTRACT-MAX(A)
1. if A.heap-size < 1
2. then error "heap underflow"
3. max = A[1]
4. A[1] = A[A.heap-size]
5. A.heap-size = A.heap-size - 1
6. MAX-HEAPIFY(A, 1)
7. return max Running time: O(lg n)
35
Example: HEAP-EXTRACT-MAX
16 1

14 10 max = 16 14 10
8 7 9 3 8 7 9 3
2 4 1 2 4
Heap size decreased with 1

14

Call MAX-HEAPIFY(A, 1)
8 10
4 7 9 3
2 1

36
HEAP-INCREASE-KEY
• Goal: Increases the key of an element i in the heap
• Idea:
– Increment the key of A[i] to its new value
– If the max-heap property does not hold anymore:
traverse a path toward the root to find the proper place
for the newly increased key 16

14 10

8 i 7 9 3

A[i] ← 15 2 4 1
37
HEAP-INCREASE-KEY
HEAP-INCREASE-KEY(A, i, key)
1. if key < A[i]
2. then error “new key is smaller than current key”
3. A[i] = key
4. while i > 1 and A[PARENT(i)] < A[i]
5. do exchange A[i] with A[PARENT(i)]
6. i = PARENT(i)

• Running time: O(lg n)

38
 Example: HEAP-INCREASE-KEY
16 16

14 10 14 10
8 i 7 9 3 8 i 7 9 3
2 4 1 2 15 1

A[i ] ← 15

16 16
i
14 10 15 10
i
15 7 9 3 14 7 9 3
2 8 1 2 8 1

39
MAX-HEAP-INSERT
16

• Goal: Inserts a new element into 14 10

a max-heap 8 7 9 3
2 4 1 -
• Idea:
– Expand the max-heap with a new
16
element whose key is -
– Calls HEAP-INCREASE-KEY to set 14 10

the key of the new node to its 8 7 9 3

2 4 1 15
correct value and maintain the
max-heap property
40
MAX-HEAP-INSERT

MAX-HEAP-INSERT(A, key)

1. A.heap-size = A.heap-size + 1

2. A[A.heap-size] = -

3. HEAP-INCREASE-KEY(A, A.heap-size, key)

Running time: O(lg n)

41
Example: MAX-HEAP-INSERT
Insert value 15: Increase the key to 15
- Start by inserting - Call HEAP-INCREASE-KEY on A[11] = 15
16 16

14 10 14 10
8 7 9 3 8 7 9 3
2 4 1 - 2 4 1 15

The restored heap containing

the newly added element

16 16

14 10 15 10

8 15 9 3 8 14 9 3

2 4 1 7 2 4 1 7
42
Create Max-heap using Max-Heap-Insert
A = [5, 3, 17, 10, 84, 19, 6, 22, 9]
1 1 1
Insert 5 Insert 3 Insert 17
5 2
5 17
2 3
A = [5] 3 A = [5, 3] 3 5
A = [17, 3, 5]

1 1
Insert 10 17 Insert 84 84
2 3 2 3
10 5 17 5
4 4 5
3 A = [17, 10, 5, 3] 3 10
A = [84, 17, 5, 3, 10]

44
A = [5, 3, 17, 10, 84, 19, 6, 22, 9] Contd…
1 1
Insert 19 Insert 6
84 84
2 3 2 3
17 19 17 19
4 5 6 4 5 6 7
3 10 5 3 10 5 6
A = [84, 17, 19, 3, 10, 5] A = [84, 17, 19, 3, 10, 5, 6]
1
Insert 22 84
2 3
22 19
4 5 6 7
17 10 5 6
8
3 A = [84, 22, 19, 17, 10, 5, 6, 3]
45
A = [5, 3, 17, 10, 84, 19, 6, 22, 9] Contd…

1
Insert 9
84
2 3
22 19
4 5 6 7
17 10 5 6
8 9
3 9

A = [84, 22, 19, 17, 10, 5, 6, 3, 9]

46
PRIO*R**I*T*Y***QUE***U*E
Character → Max-Heap-Insert. * → Extract-Max.
1 1 1
Insert P Insert R Insert I
P 2
R R
2 3
A = [P] P A = [R, P] P I
A = [R, P, I]

1 1
Insert O R Extract-Max P
2 3 2 3
P I O I
4
O A = [R, P, I, O] A = [P, O, I]
Output: R
48
PRIO*R**I*T*Y***QUE***U*E
Contd…
1 1
Insert R R Extract-Max P
2 3 2 3
P I O I
4
O A = [R, P, I, O] A = [P, O, I]
Output: R R

1 1
Extract-Max Insert I
2
O O
2 3
I A = [O, I] I I
Output: R R P
A = [O, I, I]

49
PRIO*R**I*T*Y***QUE***U*E
Contd…
1 1
Extract-Max
2
I Insert T T
2 3
I I I
A = [I, I]
Output: R R P O A = [T, I, I]

1 1
Extract-Max
2
I Insert Y Y
2 3
I I I
A = [I, I]
Output: R R P O T A = [T, I, I]

50
PRIO*R**I*T*Y***QUE***U*E
Contd…
1
1
Extract-Max I Extract-Max Extract-Max NULL
2 I
I A = [I] A = []
Output: R R P O T Y I Output: R R P O T Y I I
A = [I, I]
Output: R R P O T Y

1 1 1
Insert Q Insert U Insert E
Q 2
U U
2 3
A = [Q] Q A = [U, Q] Q E
A = [U, Q, E]

51
PRIO*R**I*T*Y***QUE***U*E
Contd…
1
1
Extract-Max Q Extract-Max Extract-Max NULL
2 E
E A = []
A = [E] Output: R R P O T Y I I U Q E
Output: R R P O T Y I I U Q
A = [Q, E]
Output: R R P O T Y I I U

1 1
Insert U Extract-Max NULL Insert E
U E
A = []
A = [U] Output: R R P O T Y I I U Q E U A = [E]

52
Applications
• Heap sort
• Priority queues: Query for minimum or maximum value in a dynamic
collection of values.
• Dijkstra's algorithm for finding the shortest path between a pair of nodes
uses heap to pick the closest unexplored node at each iteration to continue
the search from it.
Example: routing of network packets between two nodes.
• Prim's algorithm for finding the Minimum Spanning Tree uses heap to select
a new minimum-cost edge that expands your current minimum spanning
tree.
Example: wire layout for a service network, such as electricity or cable. Aim
is to provide service coverage to an entire area with the minimum wiring
cost possible.
• Huffman encoding (data compression).
• Used by an operating system for dynamic memory allocation.
58