Heapsort

Design and Analysis of Algorithms

 Special Types of Trees
• Def: Full binary tree = a 4

binary tree in which each 1 3

node is either a leaf or has
2 16 9 10
degree exactly 2. 14 8 7
12
Full binary tree

4
• Def: Complete binary tree = a
binary tree in which all leaves 1 3

are on the same level and all 2 16 9 10

internal nodes have degree 2.
Complete binary tree

2
 Definitions
• Height of a node = the number of edges on the longest
simple path from the node down to a leaf
• Level of a node = the length of a path from the root to
the node
• Height of tree = height of root node

4 Height of root = 3

1 3
Height of (2)= 1 2 16 9 10 Level of (10)= 2
14 8

3
 Useful Properties

height

height

(see Ex 6.1-2, page 129)

d +1
d
2 −1
n   2l = = 2d +1 − 1 Height of root = 3
l =0 2 −1 4

1 3
Height of (2)= 1 2 16 9 10 Level of (10)= 2
14 8

4
 The Heap Data Structure
• Def: A heap is a nearly complete binary tree with
the following two properties:
– Structural property: all levels are full, except
possibly the last one, which is filled from left to right
– Order (heap) property: for any node x
Parent(x) ≥ x

8 From the heap property, it

follows that:
7 4 “The root is the maximum
5 2 element of the heap!”
Heap
5
A heap is a binary tree that is filled in order
 Array Representation of Heaps
• A heap can be stored as an
array A.
– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]
– Heapsize[A] ≤ length[A]
• The elements in the subarray
A[(n/2+1) .. n] are leaves

6
 Heap Types
• Max-heaps (largest element at root), have the
max-heap property:
– for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]

• Min-heaps (smallest element at root), have the

min-heap property:
– for all nodes i, excluding the root:
A[PARENT(i)] ≤ A[i]

7
 Adding/Deleting Nodes
• New nodes are always inserted at the bottom
level (left to right)
• Nodes are removed from the bottom level (right
to left)

8
 Operations on Heaps
• Maintain/Restore the max-heap property
– MAX-HEAPIFY
• Create a max-heap from an unordered array
– BUILD-MAX-HEAP
• Sort an array in place
– HEAPSORT
• Priority queues

9
 Maintaining the Heap Property
• Suppose a node is smaller than a
child
– Left and Right subtrees of i are max-heaps
• To eliminate the violation:
– Exchange with larger child
– Move down the tree
– Continue until node is not smaller than
children

10
 Example
MAX-HEAPIFY(A, 2)

A[2]  A[4]

A[2] violates the heap property A[4] violates the heap property

A[4]  A[9]

Heap property restored

11
 Maintaining the Heap Property
Assumptions: Alg: MAX-HEAPIFY(A, i)
• Left and Right 1. l ← LEFT(i)
subtrees of i 2. r ← RIGHT(i)
are max-heaps 3. if l ≤ Heap-Size(A) and A[l] > A[i]
• A[i] may be 4. then largest ←l
smaller than its
5. else largest ←i
children
6. if r ≤ Heap-Size(A) and A[r] > A[largest]
7. then largest ←r
8. if largest  i
9. then exchange A[i] ↔ A[largest]
10. MAX-HEAPIFY(A, largest)

12
 MAX-HEAPIFY Running Time
• Intuitively:
– It traces a path from the root to a leaf.
– At each level it makes exactly 2 comparisons.
– Total number of comparisons is 2h.
– Running time is O(h) or O(lg n).

• Running time of MAX-HEAPIFY is O(lgn)

• Can be written in terms of the height of the heap,

as being O(h)
– Since the height of the heap is lgn 13
 Building a Heap
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2

Alg: BUILD-MAX-HEAP(A) 1

1. Heap-Size(A) = length[A] 4

for i ←  length[A] /2 downto 1

2 3
2. 1 3
4 5 6 7

3. do MAX-HEAPIFY(A, i) 8
2 9 10
16 9 10
14 8 7

A: 4 1 3 2 16 9 10 14 8 7

14
 Example: A 4 1 3 2 16 9 10 14 8 7

i=5 i=4 i=3

1 1 1

4 4 4
2 3 2 3 2 3

1 3 1 3 1 3
4 5 6 7 4 5 6 7 4 5 6 7

8
2 9 10
16 9 10 8 2 9 10
16 9 10 8 14 9 10
16 9 10
14 8 7 14 8 7 2 8 7

i=2 i=1
1 1 1

4 4 16
2 3 2 3 2 3

1 10 16 10 14 10
4 5 6 7 4 5 6 7 4 5 6 7

8
14 9 10
16 9 3 8
14 9 10
7 9 3 8
8 9 10
7 9 3
2 8 7 2 8 1 2 4 1
15
 Running Time of BUILD MAX HEAP

Alg: BUILD-MAX-HEAP(A)
1. Heap-Size(A) = length[A]
2. for i ←  length[A] /2 downto 1
O(n)
3. do MAX-HEAPIFY(A, i, n) O(lgn)

 Running time: O(nlgn)

• This is not an asymptotically tight upper bound

16
Running Time of BUILD MAX HEAP

• build-max-heap algorithm executes bottom-to-top.

• Let the size of heap = n

• Maxm no. of elements with height h = 𝑛/2ℎ+1

• When max-heapify is called to a node having height
h, the cost is O(h)

• For all nodes of height h, the total cost = 𝑛/2ℎ+1 * O(h)

17
Running Time of BUILD MAX HEAP
For all nodes with varying height, the time
complexity =
lgn
෍ 𝑛/2ℎ+1 ∗ O(h)
ℎ=0
lgn ℎ
=𝑂 𝑛 ∗ σℎ=0 ℎ
2
 ℎ
≤ 𝑂 𝑛 ∗ σℎ=0 ℎ
2
1ൗ
2
=O n∗ 2
1−1ൗ2
 𝑇 𝑛 = 𝑂(𝑛)
18
 Heapsort
• Goal:
– Sort an array using heap representations

• Idea:
– Build a max-heap from the array
– Swap the root (the maximum element) with the last
element in the array
– “Discard” this last node by decreasing the heap size
– Call MAX-HEAPIFY on the new root
– Repeat this process until only one node remains

19
Example: A=[7, 4, 3, 1, 2]

EXCHANGE A[1]  A[5] EXCHANGE A[1]  A[4] EXCHANGE A[1]  A[3]

MAX-HEAPIFY(A, 1) MAX-HEAPIFY(A, 1) MAX-HEAPIFY(A, 1)

EXCHANGE A[1]  A[2]

MAX-HEAPIFY(A, 1)

20
 Alg: HEAPSORT(A)

1. BUILD-MAX-HEAP(A) O(n)

2. for i ← length[A] downto 2

3. do exchange A[1] ↔ A[i] (1)
n-1
4. Heap-Size(A) = Heap-Size(A) -1 (1) times

5. MAX-HEAPIFY(A, 1) O(lgn)

• Running time: O(nlgn)

21
Priority Queues

22
 Operations on Priority Queues

• Max-priority queues support the following

operations:
– INSERT(S, x): inserts element x into set S

– EXTRACT-MAX(S): removes and returns element of

S with largest key

– MAXIMUM(S): returns element of S with largest key

– INCREASE-KEY(S, x, k): increases value of element

x’s key to k (Assume k ≥ x’s current key value)

23
 HEAP-MAXIMUM
Goal:
– Return the largest element of the heap

Running time: O(1)

Alg: HEAP-MAXIMUM(A)
1. return A[1]
Heap A:

Heap-Maximum(A) returns 7
24
 HEAP-EXTRACT-MAX
Goal:
– Extract the largest element of the heap (i.e., return the max
value and also remove that element from the heap
Idea:
– Exchange the root element with the last
– Decrease the size of the heap by 1 element
– Call MAX-HEAPIFY on the new root, on a heap of size n-1

Heap A: Root is the largest element

25
 Example: HEAP-EXTRACT-MAX
16 1

14 10 max = 16 14 10
8 7 9 3 8 7 9 3
2 4 1 2 4
Heap size decreased with 1

14

Call MAX-HEAPIFY(A, 1)
8 10
4 7 9 3
2 1

26
 HEAP-EXTRACT-MAX
Alg: HEAP-EXTRACT-MAX(A)

1. if heap-size[A] < 1
2. then error “heap underflow”

3. max ← A[1]

4. A[1] ← A[heap-size[A]]

5. heap-size[A] ← heap-size[A] - 1

6. MAX-HEAPIFY(A, 1) remakes heap

7. return max
Running time: O(lgn) 27
 HEAP-INCREASE-KEY
• Goal:
– Increases the key of an element i in the heap
• Idea:
– Increment the key of A[i] to its new value
– If the max-heap property does not hold anymore:
traverse a path toward the root to find the proper
place for the newly increased key
16

14 10
8 i 7 9 3
Key [i] ← 15 2 4 1

28
 Example: HEAP-INCREASE-KEY
16 16

14 10 14 10
8 i 7 9 3 8 i 7 9 3
2 4 1 2 15 1

Key [i ] ← 15

16 16
i
14 10 15 10
i
15 7 9 3 14 7 9 3
2 8 1 2 8 1

29
 HEAP-INCREASE-KEY
Alg: HEAP-INCREASE-KEY(A, i, key)

1. if key < A[i]

2. then error “new key is smaller than current key”
3. A[i] ← key
4. while i > 1 and A[PARENT(i)] < A[i] 16
5. do exchange A[i] ↔ A[PARENT(i)]
14 10
6. i ← PARENT(i)
8 i 7 9 3
2 4 1
• Running time: O(lgn)
Key [i] ← 15

30
 MAX-HEAP-INSERT
• Goal:
16
– Inserts a new element into a max-
heap 14 10
8 7 9 3
• Idea:
2 4 1 -
– Expand the max-heap with a new
16
element whose key is -
– Calls HEAP-INCREASE-KEY to 14 10
set the key of the new node to its 8 7 9 3
correct value and maintain the 2 4 1 15

max-heap property

31
 Example: MAX-HEAP-INSERT
Insert value 15: Increase the key to 15
- Start by inserting - Call HEAP-INCREASE-KEY on A[11] = 15
16 16

14 10 14 10
8 7 9 3 8 7 9 3
2 4 1 - 2 4 1 15

The restored heap containing

the newly added element

16 16

14 10 15 10

8 15 9 3 8 14 9 3

2 4 1 7 2 4 1 7
32
 MAX-HEAP-INSERT

16
Alg: MAX-HEAP-INSERT(A, key)
14 10
1. heap-size[A] ← heap-size[A] + 1 8 7 9 3
2 4 1 -
2. A[heap-size[A]] ← -

3. HEAP-INCREASE-KEY(A, heap-size[A] , key)

Running time: O(lgn)

33
 Summary
• We can perform the following operations on
heaps:
– MAX-HEAPIFY O(lgn)
– BUILD-MAX-HEAP O(n)
– HEAP-SORT O(nlgn)
– MAX-HEAP-INSERT O(lgn)
– HEAP-EXTRACT-MAX O(lgn)
Average
– HEAP-INCREASE-KEY O(lgn) O(lgn)
– HEAP-MAXIMUM O(1)
34