0% found this document useful (0 votes)
70 views31 pages

Heapsort: Comp 122, Spring 2004

Heapsort combines aspects of merge sort and insertion sort. It sorts in place like insertion sort, with a running time of O(n log n) like merge sort. It uses a binary heap data structure to manage information during the algorithm. The heap is used to sort an array by first converting it to a max-heap, then repeatedly removing the maximum element and allowing the heap to reorder. Converting the array to a max-heap takes O(n) time, and each of the n removals takes O(log n) time, giving heapsort an overall time complexity of O(n log n).

Uploaded by

Ioannis Milas
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
70 views31 pages

Heapsort: Comp 122, Spring 2004

Heapsort combines aspects of merge sort and insertion sort. It sorts in place like insertion sort, with a running time of O(n log n) like merge sort. It uses a binary heap data structure to manage information during the algorithm. The heap is used to sort an array by first converting it to a max-heap, then repeatedly removing the maximum element and allowing the heap to reorder. Converting the array to a max-heap takes O(n) time, and each of the n removals takes O(log n) time, giving heapsort an overall time complexity of O(n log n).

Uploaded by

Ioannis Milas
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 31

Heapsort

Comp 122, Spring 2004


Heapsort
 Combines the better attributes of merge sort
and insertion sort.
» Like merge sort, but unlike insertion sort, running
time is O(n lg n).
» Like insertion sort, but unlike merge sort, sorts in
place.
 Introduces an algorithm design technique
» Create data structure (heap) to manage information
during the execution of an algorithm.
 The heap has other applications beside sorting.
» Priority Queues
heapsort - 2 Comp 122 Lin / Devi
Data Structure Binary Heap
 Array viewed as a nearly complete binary tree.
» Physically – linear array.
» Logically – binary tree, filled on all levels (except lowest.)
 Map from array elements to tree nodes and vice versa
» Root – A[1]
» Left[i] – A[2i]
» Right[i] – A[2i+1]
» Parent[i] – A[i/2]
 length[A] – number of elements in array A.
 heap-size[A] – number of elements in heap stored in A.
» heap-size[A]  length[A]

heapsort - 3 Comp 122 Lin / Devi


Heap Property (Max and Min)
 Max-Heap
» For every node excluding the root,
value is at most that of its parent: A[parent[i]]  A[i]
 Largest element is stored at the root.
 In any subtree, no values are larger than the value
stored at subtree root.
 Min-Heap
» For every node excluding the root,
value is at least that of its parent: A[parent[i]]  A[i]
 Smallest element is stored at the root.
 In any subtree, no values are smaller than the value
stored at subtree root

heapsort - 4 Comp 122 Lin / Devi


Heaps – Example
26 24 20 18 17 19 13 12 14 11 Max-heap as an
1 2 3 4 5 6 7 8 9 10
array.

Max-heap as a binary
tree. 26

24 20

18 17 19 13

12 14 11 Last row filled from left to right.


heapsort - 5 Comp 122 Lin / Devi
Height
 Height of a node in a tree: the number of edges on the
longest simple downward path from the node to a leaf.
 Height of a tree: the height of the root.
 Height of a heap: lg n 
» Basic operations on a heap run in O(lg n) time

heapsort - 6 Comp 122 Lin / Devi


Heaps in Sorting
 Use max-heaps for sorting.
 The array representation of max-heap is not sorted.
 Steps in sorting
» Convert the given array of size n to a max-heap (BuildMaxHeap)
» Swap the first and last elements of the array.
• Now, the largest element is in the last position – where it belongs.
• That leaves n – 1 elements to be placed in their appropriate locations.
• However, the array of first n – 1 elements is no longer a max-heap.
• Float the element at the root down one of its subtrees so that the array
remains a max-heap (MaxHeapify)
• Repeat step 2 until the array is sorted.

heapsort - 7 Comp 122 Lin / Devi


Heap Characteristics
 Height = lg n
 No. of leaves = n/2
 No. of nodes of
height h  n/2h+1

heapsort - 8 Comp 122 Lin / Devi


Maintaining the heap property
 Suppose two subtrees are max-heaps,
but the root violates the max-heap
property.

 Fix the offending node by exchanging the value at the


node with the larger of the values at its children.
» May lead to the subtree at the child not being a heap.
 Recursively fix the children until all of them satisfy the
max-heap property.

heapsort - 9 Comp 122 Lin / Devi


MaxHeapify – Example
MaxHeapify(A, 2)
26

24
14 20

18
14
24 17 19 13

12 14
18 11

heapsort - 10 Comp 122 Lin / Devi


Procedure MaxHeapify
MaxHeapify(A, i)
1. l  left(i) Assumption:
2. r  right(i) Left(i) and Right(i)
3. if l  heap-size[A] and A[l] > A[i] are max-heaps.
4. then largest  l
5. else largest  i
6. if r  heap-size[A] and A[r] > A[largest]
7. then largest  r
8. if largest i
9. then exchange A[i]  A[largest]
10. MaxHeapify(A, largest)

heapsort - 11 Comp 122 Lin / Devi


Running Time for MaxHeapify
MaxHeapify(A, i)
1. l  left(i)
2. r  right(i)
3. if l  heap-size[A] and A[l] > A[i]
Time to fix node i and
4. then largest  l its children = (1)
5. else largest  i
6. if r  heap-size[A] and A[r] > A[largest]
PLUS
7. then largest  r
8. if largest i
Time to fix the
9. then exchange A[i]  A[largest] subtree rooted at one
10. MaxHeapify(A, largest) of i’s children =
T(size of subree at
largest)
heapsort - 12 Comp 122 Lin / Devi
Running Time for MaxHeapify(A, n)
 T(n) = T(largest) + (1)
 largest  2n/3 (worst case occurs when the last row of
tree is exactly half full)
 T(n)  T(2n/3) + (1)  T(n) = O(lg n)
 Alternately, MaxHeapify takes O(h) where h is the
height of the node where MaxHeapify is applied

heapsort - 13 Comp 122 Lin / Devi


Building a heap
 Use MaxHeapify to convert an array A into a max-heap.
 How?
 Call MaxHeapify on each element in a bottom-up
manner.

BuildMaxHeap(A)
1. heap-size[A]  length[A]
2. for i  length[A]/2 downto 1
3. do MaxHeapify(A, i)

heapsort - 14 Comp 122 Lin / Devi


BuildMaxHeap – Example
Input Array:

24 21 23 22 36 29 30 34 28 27

Initial Heap:
(not max-heap) 24

21 23

22 36 29 30

34 28 27
heapsort - 15 Comp 122 Lin / Devi
BuildMaxHeap – Example
MaxHeapify(10/2 = 5)
MaxHeapify(4)
MaxHeapify(3) 24
36
MaxHeapify(2)
MaxHeapify(1)
21
34
24
36 30
23

28
34
22
24 36
27
21 29 30
23

34
22 28
24 27
21

heapsort - 16 Comp 122 Lin / Devi


Correctness of BuildMaxHeap
 Loop Invariant: At the start of each iteration of the for
loop, each node i+1, i+2, …, n is the root of a max-heap.
 Initialization:
» Before first iteration i = n/2
» Nodes n/2+1, n/2+2, …, n are leaves and hence roots of
max-heaps.
 Maintenance:
» By LI, subtrees at children of node i are max heaps.
» Hence, MaxHeapify(i) renders node i a max heap root (while
preserving the max heap root property of higher-numbered
nodes).
» Decrementing i reestablishes the loop invariant for the next
iteration.

heapsort - 17 Comp 122 Lin / Devi


Running Time of BuildMaxHeap
 Loose upper bound:
» Cost of a MaxHeapify call  No. of calls to MaxHeapify
» O(lg n)  O(n) = O(nlg n)
 Tighter bound:
» Cost of a call to MaxHeapify at a node depends on the height,
h, of the node – O(h).
» Height of most nodes smaller than n.
» Height of nodes h ranges from 0 to lg n.
» No. of nodes of height h is n/2h+1

heapsort - 18 Comp 122 Lin / Devi


Running Time of BuildMaxHeap
Tighter Bound for T(BuildMaxHeap)

lg n 
T(BuildMaxHeap) h

h 0 2
h
lg n 
 n 
 
h 0  2
h 1 

O ( h)


h
h
, x  1 / 2 in (A.8)
h 0 2
 lg n  h 
 O n  h  
1/ 2
 h 0 2  (1  1 / 2) 2
2
 lg n  h    h
O n  h   O n  h 
 h 0 2   h 0 2 
 O ( n)

Can build a heap from an unordered array in linear time

heapsort - 19 Comp 122 Lin / Devi


Heapsort
 Sort by maintaining the as yet unsorted elements as a
max-heap.
 Start by building a max-heap on all elements in A.
» Maximum element is in the root, A[1].
 Move the maximum element to its correct final
position.
» Exchange A[1] with A[n].
 Discard A[n] – it is now sorted.
» Decrement heap-size[A].
 Restore the max-heap property on A[1..n–1].
» Call MaxHeapify(A, 1).
 Repeat until heap-size[A] is reduced to 2.
heapsort - 20 Comp 122 Lin / Devi
Heapsort(A)
HeapSort(A)
1. Build-Max-Heap(A)
2. for i  length[A] downto 2
3. do exchange A[1]  A[i]
4. heap-size[A]  heap-size[A] – 1
5. MaxHeapify(A, 1)

heapsort - 21 Comp 122 Lin / Devi


Heapsort – Example
26 24 20 18 17 19 13 12 14 11
1 2 3 4 5 6 7 8 9 10

26

24 20

18 17 19 13

12 14 11
heapsort - 22 Comp 122 Lin / Devi
Algorithm Analysis
HeapSort(A)
1. Build-Max-Heap(A)
2. for i  length[A] downto 2
3. do exchange A[1]  A[i]
 In-place 4. heap-size[A]  heap-size[A] – 1
5. MaxHeapify(A, 1)
 Not Stable

 Build-Max-Heap takes O(n) and each of the n-1 calls


to Max-Heapify takes time O(lg n).

 Therefore, T(n) = O(n lg n)

heapsort - 23 Comp 122 Lin / Devi


Heap Procedures for Sorting
 MaxHeapify O(lg n)
 BuildMaxHeap O(n)
 HeapSort O(n lg n)

heapsort - 24 Comp 122 Lin / Devi


Priority Queue
 Popular & important application of heaps.
 Max and min priority queues.
 Maintains a dynamic set S of elements.
 Each set element has a key – an associated value.
 Goal is to support insertion and extraction efficiently.
 Applications:
» Ready list of processes in operating systems by their
priorities – the list is highly dynamic
» In event-driven simulators to maintain the list of events to be
simulated in order of their time of occurrence.

heapsort - 25 Comp 122 Lin / Devi


Basic Operations
 Operations on a max-priority queue:
» Insert(S, x) - inserts the element x into the set S
• S  S  {x}.
» Maximum(S) - returns the element of S with the largest key.
» Extract-Max(S) - removes and returns the element of S with
the largest key.
» Increase-Key(S, x, k) – increases the value of element x’s key
to the new value k.
 Min-priority queue supports Insert, Minimum, Extract-
Min, and Decrease-Key.
 Heap gives a good compromise between fast insertion
but slow extraction and vice versa.

heapsort - 26 Comp 122 Lin / Devi


Heap Property (Max and Min)
 Max-Heap
» For every node excluding the root,
value is at most that of its parent: A[parent[i]]  A[i]
 Largest element is stored at the root.
 In any subtree, no values are larger than the value
stored at subtree root.
 Min-Heap
» For every node excluding the root,
value is at least that of its parent: A[parent[i]]  A[i]
 Smallest element is stored at the root.
 In any subtree, no values are smaller than the value
stored at subtree root

heapsort - 27 Comp 122 Lin / Devi


Heap-Extract-Max(A)
Implements the Extract-Max operation.

Heap-Extract-Max(A)
1. if heap-size[A] < 1
2. then error “heap underflow”
3. max  A[1]
4. A[1]  A[heap-size[A]]
5. heap-size[A]  heap-size[A] - 1
6. MaxHeapify(A, 1)
7. return max

Running time : Dominated by the running time of MaxHeapify


= O(lg n)

heapsort - 28 Comp 122 Lin / Devi


Heap-Insert(A, key)
Heap-Insert(A, key)
1. heap-size[A]  heap-size[A] + 1
2. i  heap-size[A]
4. while i > 1 and A[Parent(i)] < key
5. do A[i]  A[Parent(i)]
6. i  Parent(i)
7. A[i]  key

Running time is O(lg n)


The path traced from the new leaf to the root has
length O(lg n)

heapsort - 29 Comp 122 Lin / Devi


Heap-Increase-Key(A, i, key)
Heap-Increase-Key(A, i, key)
1 If key < A[i]
2 then error “new key is smaller than the current key”
3 A[i]  key
4 while i > 1 and A[Parent[i]] < A[i]
5 do exchange A[i]  A[Parent[i]]
6 i  Parent[i]

Heap-Insert(A, key)
1 heap-size[A]  heap-size[A] + 1
2 A[heap-size[A]]  –
3 Heap-Increase-Key(A, heap-size[A], key)

heapsort - 30 Comp 122 Lin / Devi


Examples

heapsort - 31 Comp 122 Lin / Devi

You might also like