Heapsort

Heapsort
 Combines the better attributes of merge sort
and insertion sort.
» Like merge sort, but unlike insertion sort, running
time is O(n lg n).
» Like insertion sort, but unlike merge sort, sorts in
place.
 Introduces an algorithm design technique
» Create data structure (heap) to manage information
during the execution of an algorithm.
 The heap has other applications beside sorting.
» Priority Queues
 Data Structure Binary Heap
 Array viewed as a nearly complete binary tree.
» Physically – linear array.
» Logically – binary tree, filled on all levels (except lowest.)
 Map from array elements to
tree nodes and vice versa
» Root – A[1]
» Left[i] – 2i
» Right[i] – 2i+1
» Parent[i] – i/2

 length[A] – number of elements in array A.

 heap-size[A] – number of elements in heap stored in A.
» heap-size[A]  length[A]
 Heap Property (Max and Min)
 Max-Heap
» For every node excluding the root,
value is at most that of its parent: A[parent[i]]  A[i]
 Largest element is stored at the root.
 In any subtree, no values are larger than the value
stored at subtree root.
 Min-Heap
» For every node excluding the root,
value is at least that of its parent: A[parent[i]]  A[i]
 Smallest element is stored at the root.
 In any subtree, no values are smaller than the value
stored at subtree root
 Heaps – Example
26 24 20 18 17 19 13 12 14 11 Max-heap as an
1 2 3 4 5 6 7 8 9 10
array.

Max-heap as a binary
tree. 26

24 20

18 17 19 13

12 14 11 Last row filled from left to right.

Heap or Not?
Heap or Not?
Heap or Not?
 Height
 Height of a node in a tree: the number of edges on the
longest simple downward path from the node to a leaf.
 Height of a tree: the height of the root.
 Height of a heap: lg n 
» Basic operations on a heap run in O(lg n) time
 Heaps in Sorting
 Use max-heaps for sorting.
 The array representation of max-heap is not sorted.
 Steps in sorting
» Convert the given array of size n to a max-heap (BuildMaxHeap)
» Swap the first and last elements of the array.
• Now, the largest element is in the last position – where it belongs.
• That leaves n – 1 elements to be placed in their appropriate locations.
• However, the array of first n – 1 elements is no longer a max-heap.
• Float the element at the root down one of its subtrees so that the array
remains a max-heap (MaxHeapify)
• Repeat step 2 until the array is sorted.
 Heap Characteristics
 Height = lg n
 No. of leaves = n/2
 No. of nodes of
height h  n/2h+1
 Maintaining the heap property
 Suppose two subtrees are max-heaps,
but the root violates the max-heap
property.

 Fix the offending node by exchanging the value at the

node with the larger of the values at its children.
» May lead to the subtree at the child not being a heap.
 Recursively fix the children until all of them satisfy the
max-heap property.
 MaxHeapify – Example
MaxHeapify(A, 2) 26
26 18 20
4 20
4 17 19 13
18 17 19 13
12 14 11
12 14 11
26
18 20

14 17 19 13

12 4 11
 Procedure MaxHeapify
MaxHeapify(A, i)
1. l  left(i) Assumption:
2. r  right(i) Left(i) and Right(i)
3. if l  heap-size[A] and A[l] > A[i] are max-heaps.
4. then largest  l
5. else largest  i
6. if r  heap-size[A] and A[r] > A[largest]
7. then largest  r
8. if largest i
9. then exchange A[i]  A[largest]
10. MaxHeapify(A, largest)
 Running Time for MaxHeapify
MaxHeapify(A, i)
1. l  left(i)
2. r  right(i)
3. if l  heap-size[A] and A[l] > A[i]
Time to fix node i and
4. then largest  l its children = (1)
5. else largest  i
6. if r  heap-size[A] and A[r] > A[largest]
PLUS
7. then largest  r
8. if largest i
Time to fix the
9. then exchange A[i]  A[largest] subtree rooted at one
10. MaxHeapify(A, largest) of i’s children =
T(size of subree at
largest)
Running Time for MaxHeapify(A, n)
 T(n) = T(largest) + (1)
 largest  2n/3 (worst case
occurs when the last row of
tree is exactly half full)
 T(n)  T(2n/3) + (1) 
T(n) = O(lg n)

 Alternately, MaxHeapify takes O(h) where h is the

height of the node where MaxHeapify is applied
 Building a heap
 Use MaxHeapify to convert an array A into a max-heap.
 Call MaxHeapify on each element in a bottom-up
manner.

BuildMaxHeap(A)
1. heap-size[A]  length[A]
2. for i  length[A]/2 downto 1
3. do MaxHeapify(A, i)
 BuildMaxHeap – Example
Input Array:

24 21 23 22 36 29 30 34 28 27

Initial Heap:
(not max-heap) 24

21 23

22 36 29 30

34 28 27
 BuildMaxHeap – Example
MaxHeapify(10/2 = 5)
MaxHeapify(4)
MaxHeapify(3) 24
36
MaxHeapify(2)
MaxHeapify(1)
21
34
24
36 30
23

28
34
22
24 36
27
21 29 30
23

34
22 28
24 27
21
 Running Time of BuildMaxHeap
 Loose upper bound:
» Cost of a MaxHeapify call  No. of calls to MaxHeapify
» O(lg n)  O(n) = O(nlg n)
 Tighter bound:
» Cost of a call to MaxHeapify at a node depends on the height,
h, of the node – O(h).
» Height of most nodes smaller than lg n.
» Height of nodes h ranges from 0 to lg n.
» No. of nodes of height h is n/2h+1
Running Time of BuildMaxHeap
Tighter Bound for T(BuildMaxHeap)
T(BuildMaxHeap)
lg n  lg n 
 n  h
 
h 0  2
h 1  O ( h ) 
h 0 2 h


h
 h
lg n 
 h , x  1 / 2 in (A.8)
 O n  h
 h 0 2 
 h 0 2

1/ 2
  h 
lg n
 
h   (1  1 / 2) 2
O n  h   O n  h 
 h 0 2   h 0 2   2
 O ( n)

Can build a heap from an unordered array in linear time

 Heapsort
 Sort by maintaining the as yet unsorted elements as a
max-heap.
 Start by building a max-heap on all elements in A.
» Maximum element is in the root, A[1].
 Move the maximum element to its correct final
position.
» Exchange A[1] with A[n].
 Discard A[n] – it is now sorted.
» Decrement heap-size[A].
 Restore the max-heap property on A[1..n–1].
» Call MaxHeapify(A, 1).
 Repeat until heap-size[A] is reduced to 2.
 Heapsort(A)
HeapSort(A)
1. Build-Max-Heap(A)
2. for i  length[A] downto 2
3. do exchange A[1]  A[i]
4. heap-size[A]  heap-size[A] – 1
5. MaxHeapify(A, 1)
 Heapsort – Example
26 24 20 18 17 19 13 12 14 11
1 2 3 4 5 6 7 8 9 10

26

24 20

18 17 19 13

12 14 11
Heapsort – Example

BuildMaxHeap(A)
1.heap-size[A]  length[A]
2.for i  length[A]/2 downto 1
3. do MaxHeapify(A, i)

Applying BuildMaxHeap(A)
heap-size[A] =10
i = 10/2 = 5 to 1
BuildMaxHeap(A)
 BuildMaxHeap(A)

Now we have a max heap.

16 14 10 8 7 9 3 2 4 1
1 2 3 4 5 6 7 8 9 10
 HeapSort(A)
Now we have a max heap.
16 14 10 8 7 9 3 2 4 1
1 2 3 4 5 6 7 8 9 10

i = 10 to 2

HeapSort(A)
1. Build-Max-Heap(A)
2. for i  length[A] downto 2
3. do exchange A[1]  A[i]
4. heap-size[A]  heap-size[A] – 1
5. MaxHeapify(A, 1)
HeapSort(A)
HeapSort(A)
 HeapSort(A)

Now we have a sorted array.

1 2 3 4 7 8 9 10 14 16
1 2 3 4 5 6 7 8 9 10
 Algorithm Analysis
HeapSort(A)
1. Build-Max-Heap(A)
2. for i  length[A] downto 2
3. do exchange A[1]  A[i]
 In-place 4. heap-size[A]  heap-size[A] – 1
5. MaxHeapify(A, 1)
 Not Stable

 Build-Max-Heap takes O(n) and each of the n-1 calls

to Max-Heapify takes time O(lg n).

 Therefore, T(n) = O(n lg n)

 Heap Procedures for Sorting
 MaxHeapify O(lg n)
 BuildMaxHeap O(n)
 HeapSort O(n lg n)
 Priority Queue
 Popular & important application of heaps.
 Max and min priority queues.
 Maintains a dynamic set S of elements.
 Each set element has a key – an associated value.
 Goal is to support insertion and extraction efficiently.
 Applications:
» Ready list of processes in operating systems by their
priorities – the list is highly dynamic
» In event-driven simulators to maintain the list of events to be
simulated in order of their time of occurrence.
 Basic Operations
 Operations on a max-priority queue:
» Insert(S, x) - inserts the element x into the set S
• S  S  {x}.
» Maximum(S) - returns the element of S with the largest key.
» Extract-Max(S) - removes and returns the element of S with
the largest key.
» Increase-Key(S, x, k) – increases the value of element x’s key
to the new value k.
 Min-priority queue supports Insert, Minimum, Extract-
Min, and Decrease-Key.
 Heap gives a good compromise between fast insertion
but slow extraction and vice versa.
 Heap Property (Max and Min)
 Max-Heap
» For every node excluding the root,
value is at most that of its parent: A[parent[i]]  A[i]
 Largest element is stored at the root.
 In any subtree, no values are larger than the value
stored at subtree root.
 Min-Heap
» For every node excluding the root,
value is at least that of its parent: A[parent[i]]  A[i]
 Smallest element is stored at the root.
 In any subtree, no values are smaller than the value
stored at subtree root
 Heap-Extract-Max(A)
Implements the Extract-Max operation.

Heap-Extract-Max(A)
1. if heap-size[A] < 1
2. then error “heap underflow”
3. max  A[1]
4. A[1]  A[heap-size[A]]
5. heap-size[A]  heap-size[A] - 1
6. MaxHeapify(A, 1)
7. return max

Running time : Dominated by the running time of MaxHeapify

= O(lg n)
 Heap-Insert(A, key)
Heap-Insert(A, key)
1. heap-size[A]  heap-size[A] + 1
2. i  heap-size[A]
4. while i > 1 and A[Parent(i)] < key
5. do A[i]  A[Parent(i)]
6. i  Parent(i)
7. A[i]  key

Running time is O(lg n)

The path traced from the new leaf to the root has
length O(lg n)
 Heap-Increase-Key(A, i, key)
Heap-Increase-Key(A, i, key)
1 If key < A[i]
2 then error “new key is smaller than the current key”
3 A[i]  key
4 while i > 1 and A[Parent[i]] < A[i]
5 do exchange A[i]  A[Parent[i]]
6 i  Parent[i]

Heap-Insert(A, key)
1 heap-size[A]  heap-size[A] + 1
2 A[heap-size[A]]  –
3 Heap-Increase-Key(A, heap-size[A], key)
Example: Heap-Insert(A, 15)
Example:Heap-Increase-Key(A, 9, 15)