CSE-209

Algorithms-I

Lecture 6

Sorting: Heap Sort

Md. Rafsan Jani

Assistant Professor
Department of Computer Science and Engineering
Jahangirnagar University
 Priority Queue
A data structure implementing a set S of elements, each
associated with a key, supporting the following
operations:

insert(S, x) : insert element x into set S

max(S) : return element of S with largest key
extract_max(S) : return element of S with largest
key and remove it from S
increase_key(S, x, k) : increase the value of element x’ s key
to new value k
(assumed to be as large as current value)

2
 Binary Heap
● The binary heap is the classic method used to implement
priority queues.
● We use the term heap to refer to the binary heap.
● Heap is different from the term heap used in dynamic
memory allocation.
● Heap has two properties:
– Structure property
– Ordering property

3
 Structure Property

● A heap is a complete binary tree, represented as

an array.
● A complete binary tree is a tree that is
completely filled, with the possible exception of
the bottom level, which is filled from left to right.

4
Properties of a complete binary tree

● A complete binary tree of height h has between 2h

and 2h+1 – 1 nodes
● The height of a complete binary tree is
⎣log N⎦.
● It can be implemented as an array such that:
– For any element in array position i :
• the left child is in position 2i,
• the right child is in the cell after the left child (2i + 1), and
• the parent is in position ⎣ i/2 ⎦.

5
Figure 21.1
A complete binary tree and its array representation

6
 Heap-Order Property
● In a heap, for every node X with parent P, the key in P is
smaller than or equal to the key in X.
● Thus the minimum element is always at the root.
– Thus we get the extra operation findMin in constant time.
● A max heap supports access of the maximum element
instead of the minimum, by changing the heap property
slightly.

7
Figure 21.3
Two complete trees: (a) a heap; (b) not a heap

8
 Heap

•Implementation of a priority queue

•An array, visualized as a nearly complete binary tree
•Max Heap Property: The key of a node is ≥ than the keys
of its children
(Min Heap defined analogously)

2 16 1 3
14 10
5 6 7 1 2 3 4 5 6
4 7 8 9 10
8 7 9 3 1614 10 8 7 9 3 2 4 1
8
9
2 410 1

9
 Heap as a Tree
root of tree: first element in the array, corresponding to i =
1 parent(i) =i/2: returns index of node's parent
left(i)=2i: returns index of node's left child
right(i)=2i+1: returns index of node's right child

2 16 1 3
14 10
5 6 7 1 2 3 4 5 6
4 7 8 9 10
8 7 9 3 1614 10 8 7 9 3 2 4 1
8
9
2 410 1

No pointers required! Heig ht of a binary heap is O(lg

5

n)
 Heap Operations

build_max_heap : produce a max-heap from an unordered

array

max_heapify : correct a single violation of the heap

property in a subtree at its root

insert, extract_max, heapsort

11
 Max_heapify
• Assumethat the trees rooted at left(i) and right(i) are
max-heaps

• If
element A[i] violates the max-heap property, correct
violation by “trickling” element A[i] down the tree, making
the subtree rooted at index i a max-heap

12
 Max_heapify (Example)

Node 10 is the left child of node 5 but is

13
drawn to the right for convenience
Max_heapify (Example)

14
 Max_heapify (Example)

Time=? O(log
n) 15
 Max_Heapify Pseudocode

l = left(i)
r = right(i)
if (l <= heap-size(A) and A[l] >
A[i]) then largest = l
else largest = i
if (r <= heap-size(A) and A[r] >
A[largest]) then largest = r
if largest = i
then exchange A[i] and
16

A[largest] Max_Heapify(A,
 Build_Max_Heap(A)

Converts A[1…n] to a max heap

Build_Max_Heap(A):
for i=n/2 downto 1
do Max_Heapify(A, i)

Why start at n/2?

Because elements A[n/2 + 1 … n] are all leaves of the

tree 2i > n, for i > n/2 + 1

Time=? O(n log n) via simple analysis

17
 Build_Max_Heap(A) Analysis

Converts A[1…n] to a max heap

Build_Max_Heap(A):
for i=n/2 downto 1
do Max_Heapify(A, i)

Observe however that Max_Heapify takes O(1) for

time for nodes that are one level above the leaves, and
in general, O(l) for the nodes that are l levels above the
leaves. We have n/4 nodes with level 1, n/8 with level
2, and so on till we have one root node that is lg n levels
above the leaves.
18
 Build_Max_Heap(A) Analysis

Converts A[1…n] to a max heap

Build_Max_Heap(A):
for i=n/2 downto 1
do Max_Heapify(A, i)

Total amount of work in the for loop can be summed

as: n/4 (1 c) + n/8 (2 c) + n/16 (3 c) + … + 1 (lg n c)
Setting n/4 = 2k and simplifying we get:
c 2k( 1/20 + 2/21 + 3/22 + … (k+1)/2k ) The
term in brackets is bounded by a constant!

This means that Build_Max _ Heap is O(n)

1 4
Build-Max-Heap Demo

20
Build-Max-Heap Demo

21
Build-Max-Heap

22
 Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

23
 Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the array!

24
 Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the array!
4. Discard node n from heap
(by decrementing heap-size variable)

25
 Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the array!
4. Discard node n from heap
(by decrementing heap-size variable)
5. New root may violate max heap property, but its
children are max heaps. Run max_heapify to fix
this.

26
 Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the array!
4. Discard node n from heap
(by decrementing heap-size variable)
5. New root may violate max heap property, but its
children are max heaps. Run max_heapify to fix
this.
6. Go to Step 2 unless heap is empty.
27
 Heap-Sort Demo

Swap A[10] and

A[1]

Max_heapify(A,1)

28
 Heap-Sort Demo

Swap A[9] and

A[1]

29
Heap-Sort Demo

30
 Heap-Sort Demo

Swap A[8] and

A[1]

31
 Heap-Sort
● Running time:
● after n iterations the Heap is empty
● every iteration involves a swap and a max_heapify operation;
hence it takes O(log n) time

Overall O(n log n)

32
 Analysis of Heapsort
● It is an O(N log N) algorithm.
– First phase: Build heap O(N)
– Second phase: N deleteMax operations: O(NlogN).
● Detailed analysis shows that, the average case for heapsort is
poorer than quick sort.
– Quicksort’s worst case however is far worse.
● An average case analysis of heapsort is very complicated, but
empirical studies show that there is little difference between
the average and worst cases.
– Heapsort usually takes about twice as long as quicksort.
– Heapsort therefore should be regarded as something of an insurance
policy:
– On average, it is more costly, but it avoids the possibility of O(N2).

33