Design and Analysis

of Algorithms
Lecture 9
Instructor: Yusra Mansoor
Email: [email protected]
Sorting
Problem Definition:

Input is usually an n-element array.

Sorting-Structure of the data
Generally numbers are not isolated values. Rather the
numbers are part of a record.
Each such record contains a key which is to be sorted.
We are not really interested in the remaining data in the
record, called the satellite data.
If the records are permuted, then the data also needs to
be permuted; if data is large, then we often just
permute pointers to the data.
We focus on the sorting problem; and assume the input
consists of only numbers.
Heap Sort
Like merge sort, it’s running time is O(nlogn)
It sorts in place.
Heap sort introduces a design technique: heap
Heap can be visualized as a complete binary tree.
Trees
Why use Trees:

Linear access time of linear linked lists is prohibitive

◦ Does there exist any simple data structure for which the
running time of most operations (search, insert, delete) is O(log
N)?
Trees
A tree is a collection of nodes
◦ The collection can be empty
◦ (recursive definition) If not empty, a tree consists of a distinguished node r (the root), and zero or more
nonempty subtrees T1, T2, ...., Tk, each of whose roots are connected by a directed edge from r
 Trees
Terminology
◦ Root  no predecessor
◦ Leaf  no successor
◦ Interior  non-leaf
◦ Height  distance from root to leaf

Root node

Interior nodes Height

Leaf nodes
Some Terminologies
Path
Length
◦ number of edges on the path

Depth of a node
◦ length of the unique path from the root to that node
◦ The depth of a tree is equal to the depth of the deepest leaf

Height of a node
◦ length of the longest path from that node to a leaf
◦ all leaves are at height 0
◦ The height of a tree is equal to the height of the root

Ancestor and descendant

◦ Proper ancestor and proper descendant
Example: UNIX Directory
 Binary Trees
A tree in which no node can have more than two children

Binary Tree
The depth of an “average” binary tree is considerably smaller than N, even
though in the worst case, the depth can be as large as N – 1.
Types of Binary Trees
Perfect- A perfect binary tree is a type of binary tree in which every
internal node has exactly two child nodes and all the leaf nodes are
at the same level.
Full - A full Binary tree is a special type of binary tree in which every
parent node/internal node has either two or no children.
Complete - A complete binary tree is a binary tree in which all the
levels are completely filled except possibly the lowest one, which is
filled from the left.
Balanced - A balanced binary tree, also referred to as a height-
balanced binary tree, is defined as a binary tree in which the height
of the left and right subtree of any node differ by not more than 1.
LST height – RST height <= 1
Complete or Not?
 Balanced binary trees
Recall:
◦ The depth of a node is its distance from the root
◦ The depth of a tree is the depth of the deepest node

A binary tree of depth d is balanced if all the nodes at depths 0 through d-2
have two children

d-2
d-1
d
Balanced Balanced Not balanced
Why study Heapsort?
It is a well-known, traditional sorting
algorithm you will be expected to know
Heapsort is always O(n log n)
◦Quicksort is usually O(n log n) but in the worst
case slows to O(n2)
◦Quicksort is generally faster, but Heapsort is
better in time-critical applications
 What is a “heap”?
Definitions of heap:
1. A large area of memory from which the programmer can
allocate blocks as needed, and deallocate them (or allow
them to be garbage collected) when no longer needed
2. A balanced, left-justified binary tree in which no node has a
value greater than the value in its parent
These two definitions have little in common
Heapsort uses the second definition
 Outline
First, how to represent an array as a binary tree.
second, we will learn how to turn a binary tree into a
heap
Next, we will learn how to turn a binary tree back into a
heap after it has been changed in a certain way
Finally we will see how to use these ideas to sort an array
Heap Sort
 The heap property
A node has the heap property (or heap condition) if the value
in the node is as large as or larger than the values in its
children

12 12 12

8 3 8 12 8 14
Blue node has Blue node has Blue node does not
heap property heap property have heap property
All leaf nodes automatically have the heap property
A binary tree is a heap if all nodes in it have the heap property
Types of Heaps
 Mapping into an array
25

22 17

19 22 14 15

18 14 21 3 9 11

1 2 3 4 5 6 7 8 9 10 11 12 13
25 22 17 19 22 14 15 18 14 21 3 9 11
Notice:
◦ The left child of index i is at index 2*i
◦ The right child of index i is at index 2*i+1
◦ Example: the children of node 3 (19) are 7 (18) and 8 (14)
Exercise
Is the array with values {23; 17; 14; 6; 13; 10; 1; 5; 7; 12}
a max-heap?

Is this array with values {1,3,5,4,6,13,10,9,8,

15,17} a max-heap?
Is the array with values {23; 17; 14; 6; 13; 10; 1; 5; 7;
12} a max-heap?
Construction
Three main procedures:
◦ HEAPIFY: to maintain the heap property
◦ BUILD_HEAP: produces a heap from an unsorted input array
◦ HEAPSORT: sorts an array in place

EXTRACT-MAX and INSERT procedures, allow the heap data structure to be used
as a priority queue
 HEAPIFY
Given a node that does not have the heap property, you can
give it the heap property by exchanging its value with the
value of the larger child

12 14

8 14 8 12
Blue node does not Blue node has
have heap property heap property

Notice that the child may have lost the heap property
A sample heap
Here’s a sample binary tree after it has been heapified

25

22 17

19 22 14 15

18 14 21 3 9 11

Notice that heapified does not mean sorted

Heapifying does not change the shape of the binary tree; this
binary tree is balanced and left-justified because it started out
that way
HEAPIFY-pseudo code
At each step, the largest
element is determined and
stored in largest variable
If A[i] is largest, the
procedure terminates
Otherwise, the A[i] is
swapped with one of it’s
children. Which might violate
the heap property.
Need to call HEAPIFY
recursively
HEAPIFY- example

HEAPIFY(A,2)
BUILD-HEAP

We run the HEAPIFY in a bottom-up manner to convert an array in

to a heap.
Each call to HEAPIFY costs O(logn) and there are O(n) such calls.
The running time is thus at most O(n log n )
Example:
 Removing the root
Notice that the largest number is now in the root
Suppose we discard the root:
11

22 17

19 22 14 15

18 14 21 3 9 11

How can we fix the binary tree so it is once again balanced

and left-justified?
Solution: remove the rightmost leaf at the deepest level and
use it for the new root
 The reHeap method I

Our tree is balanced and left-justified, but no longer a heap

However, only the root lacks the heap property
11

22 17

19 22 14 15

18 14 21 3 9

We can HEAPIFY the root

After doing this, one and only one of its children may have lost the heap
property
 The reHeap method II
Now the left child of the root (still the number 11) lacks the
heap property
22

11 17

19 22 14 15

18 14 21 3 9

We can HEAPIFY this node

After doing this, one and only one of its children may have
lost the heap property
 The reHeap method III
Now the right child of the left child of the root (still the
number 11) lacks the heap property:
22

22 17

19 11 14 15

18 14 21 3 9

We can HEAPIFY this node

After doing this, one and only one of its children may have
lost the heap property —but it doesn’t, because it’s a leaf
The reHeap method IV
Our tree is once again a heap, because every node in it has the heap
property

22

22 17

19 21 14 15

18 14 11 3 9

Once again, the largest (or a largest) value is in the root

We can repeat this process until the tree becomes empty
This produces a sequence of values in order largest to smallest
Sorting
What do heaps have to do with sorting an array?
Here’s the neat part:
◦ Because the binary tree is balanced and left justified, it can be
represented as an array
◦ All our operations on binary trees can be represented as
operations on arrays
◦ To sort:
heapify the array;
while the array isn’t empty {
remove and replace the root;
reheap the new root node;
}
Removing and replacing the root
The “root” is the first element in the array
The “rightmost node at the deepest level” is the last element
Swap them...

1 2 3 4 5 6 7 8 9 10 11 12 13
25 22 17 19 22 14 15 18 14 21 3 9 11

1 2 3 4 5 6 7 8 9 10 11 12 13
11 22 17 19 22 14 15 18 14 21 3 9 25

...And pretend that the last element in the array no longer exists—
that is, the “last index” is 12 (9)
Reheap and repeat
Reheap the root node (index 0, containing 11)...
1 2 3 4 5 6 7 8 9 10 11 12 13
11 22 17 19 22 14 15 18 14 21 3 9 25

1 2 3 4 5 6 7 8 9 10 11 12 13
22 22 17 19 21 14 15 18 14 11 3 9 25

1 2 3 4 5 6 7 8 9 10 11 12 13
9 22 17 19 22 14 15 18 14 21 3 22 25
...And again, remove and replace the root node
Remember, though, that the “last” array index is changed
Repeat until the last becomes first, and the array is sorted!
Algorithm
Algorithm & Analysis

Here’s the algorithm:

HEAPIFY takes O(log n) time

BUILD-HEAP takes O(nlogn)
Heap Sort is thus O(nlogn)
Illustrate the operation of HEAPSORT on the
array A = {5; 13; 2; 25; 7; 17; 20; 8; 4}
 Priority Queues
A priority queue is a data structure for maintaining a set S of
elements, each with an associated value called a key.