CSI2110

Data Structures
and Algorithms

1
 Heaps
• Heaps
• Properties
• Deletion, Insertion, Construction
• Implementation of the Heap
• Implementation of Priority Queue
using a Heap
• An application: HeapSort

2
 Heaps (Min-heap)
Complete binary tree that stores a collection of keys
(or key-element pairs) at its internal nodes and that satisfies
the additional property:
key(parent) £ key(child)
REMEMBER:
complete
binary tree
all levels are full,
except the
last one, which is
left-filled

3
 Max-heap

key(parent) ³ key(child)

40

35 26

15 19 17 20

1 13 14 12 11 8

4
We store the keys in the internal nodes only

15 9

16

After adding the leaves the resulting tree is full

5
 Height of a Heap
• Theorem: A heap storing n keys has height O(log n)
Proof:
– Let h be the height of a heap storing n keys
– Since there are 2i keys at depth i = 0, … , h - 2 and at least one
key at depth h - 1, we have n ³ 1 + 2 + 4 + … + 2h-2 + 1
– Thus, n ³ 2h-1 , i.e., h £ log n + 1

depth keys
0 1

1 2
h-2 2h-2
h-1 at least 1
h
6
Notice that ….

• We could use a heap to implement a priority queue

• We store a (key, element) item at each internal
node
removeMin():
à Remove the root
à Re-arrange the heap!

(2, Sue)

(5, Pat) (6, Mark)

(9, Jeff) (7, Anna)

7
 Removal From a Heap

• The removal of the top

key leaves a hole

• We need to fix the

heap

• First, replace the hole

with the last key in
the heap

• Then, begin Downheap

… 8
 Downheap

• Downheap compares the parent with the smallest

child. If the child is smaller, it switches the two.
9
Downheap Continues

10
Downheap Continues

11
 End of Downheap

• Downheap terminates when the key is greater than the

keys of both its children or the bottom of the heap is
reached. 12
 Heap Insertion
The key to insert is 6

13
Heap Insertion

14
 Upheap
• Swap parent-child keys out of order

15
Upheap Continues

16
 End of Upheap

• Upheap terminates when new key is greater than the key

of its parent or the top of the heap is reached
• (total #swaps) (h - 1), which is O(log n) 17
 Heap Construction

We could insert the Items one at the time with

a sequence of Heap Insertions:
n

S log k = O(n log n)

k=1

But we can do better ….

O(n) using Bottom-up Heap Construction

18
 Bottom-up Heap
Construction

• We can construct a heap

storing n given keys using a
bottom-up construction

19
 Construction of a Heap
Idea: Recursively re-arrange each sub-tree
in the heap starting with the leaves

6th 5th

begin here
4th 3rd 2nd 1st
begin here

HEAP

HEAP HEAP

20
 Example 1 (Max-Heap)
2

--- keys already in the tree ---

4 5

7 3 10 8 6

1 9 6 3

I am now drawing the

3«6 leaves anymore here

2 2 2

P P
4 5 4 5 4 10

P P
7 6 10 8 9 P 6 10 8 9 P 6 5 8

1 9 3 1 7 3 1 7 3

7«9 5 « 10
21
 Example 1
2

P
4 10

P
9 P 6 5 8

1 7 3

9
9
4«9
4

This is not a heap !

7
4«7
1 7
1 4

2
P P
9 10

7 6 5 8
22
1 4 3
 Example 1
2
P P
9 10
10
7 6 5 8
2
Finally: 2 « 10
1 4 3
5 8

10
2«8
2

5 8

10

9 8

7 6 5 2

1 4 3 23
--- keys given one at a time ---

Example 2 (min-heap)
[20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]

16 15 4 12 6 7 23 20

25 5 11 27

16 15 4 12 6 7 23 20

24
 Example 2
20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]

25 5 11 27

16 15 4 12 6 9 23 20

15 4 6 23

16 25 5 12 11 9 27 20

25
 Example 2
20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]

7 8

15 4 6 23

16 25 5 12 11 9 27 20

4 6

15 5 8 23

16 25 7 12 11 9 27 20

26
 Example 2
20,23,7,6,12,4,15,16,27,11,5,25,8,7,10]

10

4 6

15 5 8 23

16 25 7 12 11 9 27 20

5 6

15 7 8 23

16 25 10 12 11 9 27 20

27
 Analysis of Heap Construction

Number of swaps
h=4

3 swaps 2 level 0
2 swaps 4 5
level 1

1 swap 7 3 10 8 level 2

0 swaps 1 9 6
level 3

Let L be the max level

(L = h-1)
level i -------- L-i swaps
28
 Analysis of Heap Construction

Number of swaps
level 0
1

L
At level i the number of swaps is
≤ L–i for each node
At level i there are ≤ 2i nodes
L
Total: ≤ S(L – i)·2i
i=0

29
Calculating O(S(L – i)·2 )
i

Let j = L-i, then i = L-j and

L
L L
S(L – i)·2i = Sj 2L-j = 2L S j 2-j
j=0
i=0 j=0

Consider Sj·2-j:
S j·2-j = 1/2 + 2 1/4 + 3 1/8 + 4 1/16 + …
= 1/2 + 1/4 + 1/8 + 1/16 + … <= 1
+ 1/4 + 1/8 + 1/16 + … <= 1/2
+ + 1/8 + 1/16 + … <= 1/4

Sj·2-j <= 2
So 2L Sj 2-j <= 2.2L =2n where L is O(log 30
n)
 L

2L Sj/2
j=1
j
≤ 2L+1 = 2n

Where L is O(log n)
O(n)

So, the number of swaps is ≤ O(n)

31
 Review

• Geometric Sum : f(i) = ai

• The geometric progressions have an exponential growth
n
S = ∑ ri = 1 + r + r2 + … + rn
i=0
rS = r + r2 + … + rn + rn+1
rS - S = (r-1)S = rn+1 - 1
S = (rn+1-1)/(r-1)
If r=2, S = (2n+1-1)
Analysis of Algorithms 32
 Implementing a Heap with an
Array
A heap can be nicely represented by a vector (array-based),
where the node at rank i has 1
2 3
- left child at rank 2i 4 5 6
7
8
and

- right child at rank 2i + 1

1 2 3 4 5 6 7 8

The leaves do no need to be explicitly stored

33
 Example
1
H

2
3
D I

4 5 6 7
B E L O

8 9 10 11 12 13
A C F G H N

2i 2i+1

1 2 3 4 5 6 7 8 9 10 11 12 13

H D I B E L O A C F G H N

34
Reminder …..

Left child of T[i] T[2i] if 2i £ n

Right child of
T[2i+1] if 2i + 1 £ n
T[i]
Parent of T[i] T[i div 2] if i>1
The Root T[1] if T¹0
Leaf? T[i] TRUE if 2i > n

1 n = 11

2 3
I

4 5 6 7

8 9 10 11

35
Implementation of a Priority
Queue with a Heap

36
 (upheap)

O(log n)
O(1)
O(log n)

(remove root + downheap)

37
 Application: Sorting
Heap Sort

Construct initial heap O(n)

remove root O(1)

n re-arrange O(log n)

times remove root O(1)

re-arrange O(log (n-1))

! "
!
38
When there are i nodes left in the PQ: ëlog iû
n
àTOT = S ëlog iû
i=1

= (n + 1)q – 2q+1 + 2 where q = ëlog (n+1)û

O(n log n) The heap-sort algorithm

sorts a sequence S of n
elements in O(n log n) time

Remember it was the O(n2)

running time of selection-
sort and insertion-sort 39
 Heap Sort animation
• https://www.youtube.com/watch?v=MtQL_
ll5KhQ

40
 HeapSort in Place

Instead of using a secondary data structure P to sort a sequence S,

We can execute heapsort « in place » by dividing S in two parts, one
representing the heap, and the other representing the sequence.
The algorithm is executed in two phases:
ü Phase 1: We build a max-heap so to occupy the whole structure.
ü Phase 2: We start with the part « sequence » empty and we
grow it by removing at each step i (i=1..n) the max value from the heap
and by adding it to the part « sequence », always maintaining
the heap properties for the part « heap ».

41
 RemoveMax()
22
19 20

15 13 14 11

2 10 5 4 1 8 3 7

7
19 20

15 13 14 11

2 10 5 4 1 8 3 22

Not a heap anymore !

42
 7
19 20

15 13 14 11

2 10 5 4 1 8 3 22

43
 20
19 7

15 13 14 11

2 10 5 4 1 8 3 22

44
 20
19 14

15 13 7 11

2 10 5 4 1 8 3 22

45
 RemoveMax()
20
19 14

15 13 8 11

2 10 5 4 1 7 3 22

Now the part heap is smaller, the part Sequence

contains a single element.

46
 Restructure
3
19 14

15 13 8 11

2 10 5 4 1 7 20 22

Not a heap anymore !

47
 Restructure
19
3 14

15 13 8 11

2 10 5 4 1 7 20 22

Not a heap anymore !

48
 Restructure
19
15 14

3 13 8 11

2 10 5 4 1 7 20 22

Not a heap anymore !

49
 RemoveMax()
19
15 14

10 13 8 11

2 3 5 4 1 7 20 22

Now it is a heap again !

50
 Restructure
7
15 14

10 13 8 11

2 3 5 4 1 19 20 22

51
 Restructure
15
13 14

10 7 8 11

2 3 5 4 1 19 20 22

52
 RemoveMax()
1
13 14

10 7 8 11

2 3 5 4 15 19 20 22

53
 Restructure
14
13 11

10 7 8 1

2 3 5 4 15 19 20 22

54
 RemoveMax()
4
13 11

10 7 8 1

2 3 5 14 15 19 20 22

55
 Restructure
13
10 11

4 7 8 1

2 3 5 14 15 19 20 22

56
 13
10 11

4 7 8 1

2 3 5 14 15 19 20 22

Heap part: unsorted Sequence part: sorted

13 10 11 4 7 8 1 2 3 5 14 15 19 20 22

57
Pseudocode for in-place HEAPSORT
(based on wikipedia pseudocode)

procedure heapsort(A,n) {
input: an unordered array A of length n

heapify(A,n) // in O(n) with bottom-up heap construction

// or in O(n log n) with n heap insertions

// Loop Invariant: A[0:end] is a heap; A[end+1:n-1] is sorted

end ← n - 1
while end > 0 do
swap(A[end], A[0])
end ← end - 1
downHeap(A, 0, end)
}
58
Procedure downHeap(A, start, end) {
root ← start
while root * 2 + 1 ≤ end do (While the root has at least one child)
child ← root * 2 + 1 (Left child)
swap ← root (Keeps track of child to swap with)
if A[swap] < A[child]
swap ← child
(If there is a right child and that child is greater)
if child+1 ≤ end and A[swap] < A[child+1]
swap ← child + 1
if swap = root
(case in which we are done with downHeap)
return
else
swap(A[root], A[swap])
root ← swap (repeat to continue downHeap the child now)
}
59
procedure heapify(A, n)
(start is assigned the index in ‘A' of the last parent node)
(the last element in a 0-based array is at index n-1;
find the parent of that element)
start ← floor ((n - 2) / 2)

while start ≥ 0 do
(downHeap the node at index 'start' to the proper place)
downHeap(A, start, n - 1)
(go to the next parent node)
start ← start - 1

// after this loop array A is a heap

60