Introduction to

Algorithms

Dr. Metwally Rashad Lecture 5

2022
Table of Contents
1. Foundations
- The Role of Algorithms in Computing
- Design and Analysis algorithms
- Growth of Functions
2. Sorting
- Heapsort
- Quicksort
- Sorting in Linear Time
3. Graph Algorithms
- Elementary Graph Algorithms
- Minimum Spanning Trees
4. Advanced Design and Analysis Techniques
- Dynamic Programming
- Greedy Algorithms
5. Selected Topics
- Linear Programming
- String Matching
1/30
 Sorting Problem
• Sort a sequence of numbers into increasing order

Example:
Input: 8 2 4 9 3 6
Output:
2 3 4 6 8 9
- Input sequence is called an instance of the sorting problem. 2/30
Sorting Algorithms

3/30
Ch2: Heapsort
- Heap Structure
- Maintaining the heap property
- Building a heap
- The heapsort algorithm

4/30
 Heap Structure
 An array of objects than can be viewed as a complete binary tree
such that:
– Each tree node corresponds
to elements of the array
– The tree is complete except possibly the
lowest level, filled from left to right
– The heap property for all nodes 𝐼 in the
tree must be maintained except for the root
• Parent node(𝐼)  𝐼

5/30
 Heap Example
• Given array 22 13 10 8 7 6 2 4 3 5

• The root of the tree is A[1]

• Given index 𝑖 of a node, formula to compute parents,
children
– Parent(𝑖)
return ⌊𝑖/2⌋
– Left Child(𝑖)
return 2𝑖
– Right Child(𝑖)
return 2𝑖+1
6/30
 Heap Attributes
 A heap represented as an array 𝐴 represented has two attributes:
– Length(A) – (𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ) – gives the number of elements in the array 𝐴
– HeapSize(A)- (𝐴. ℎ𝑒𝑎𝑝-𝑠𝑖𝑧𝑒) – represents how many elements in the heap are
stored within array 𝐴
 The property
– 0 ≤ 𝐴. ℎ𝑒𝑎𝑝-𝑠𝑖𝑧𝑒 ≤ 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ.
 The heap property is stated as
– A[parent(I)]  A[I]

 We define the height of a node in a heap to be the number of edges on the

longest simple downward path from the node to a leaf, and we define the
height of the heap to be the height of its root.
7/30
 Kinds of Binary Heaps
 Max-heaps Array A 19 12 16 1 4 7
– Store data in ascending order
19
– The max-heap property is that for every node 𝑖
other than the root, 12 16
- A[Parent(i)] ≥ A[i] Max-heap
1
– The largest element in a max-heap is stored at 4 7
the root

 Min-heaps Array A 1 4 16 7 12 19
– Store data in descending order
1
– The min-heap property is that for every node 𝑖
other than the root, 4 16 Min-heap
- A[Parent(i)] ≤ A[i]
– The smallest element in a min-heap is at the root 7 12 19
8/30
 Heapsort Algorithm
o A sorting algorithm that works by first organizing the data to be sorted into a
special type of binary tree called a heap.
o For the heapsort algorithm, we use max-heaps.
 Procedures on Heap
o Heapify
 Maintain the max-heap property, we call the procedure MAX-HEAPIFY.
 How to turn a binary tree into a max-heap
o Build Heap
 Convert an array into a max-heap
o Heapsort
 Heapsort algorithm starts by using BUILD-MAX-HEAP to build a max-heap 9/30
 Maintain the Heap Property
 In order to maintain the max-heap property, we call the procedure Max-
Heapify.

 Its inputs are an array 𝐴 and an index 𝑖 into the array.

 When it is called, Max-Heapify assumes that the binary trees rooted at

LEFT(i) and RIGHT(i) are max-heaps, but that 𝐴[𝑖] might be smaller than
its children, thus violating the max-heap property.

 Max-Heapify lets the value at 𝐴[𝑖] “float down” in the max-heap so that the
subtree rooted at index 𝑖 obeys the max-heap property.
10/30
Maintaining the heap property(cont.)

11/30
 Example of Max-Heapify
A=[1 13 10 8 7 6 2 4 3 5] A=[13 1 10 8 7 6 2 4 3 5]
 Max-Heapify(A,1). 1
13
1
1 2 3
1 10
2 3
13 10 4 5 6 7

4 5 6 7
8 7 6 2

8 7 6 2 8 9 10

8 9 10
4 3 5

4 3 5  Next is Max-Heapify(A,2). 12/30

 Example of Max-Heapify(cont.)
A=[13 8 10 1 7 6 2 4 3 5] A=[13 8 10 4 7 6 2 1 3 5]
 Next is Max-Heapify(A,4) 1
13
1
13
2 3
8 10
2 3
8 10 4 5 6 7

4 5 6 7 4 7 6 2

1 7 6 2 8 9 10 On this iteration
we have reached
8 9 10
1 3 5 a leaf and are
finished.
4 3 5
 Next is Max-Heapify(A,8). 13/30
 Max-Heapify Runtime
 Conservative recurrence for runtime:
2n
T ( n)  T ( )  (1) 𝑻 (𝒏) = 𝑶(𝒍𝒈 𝒏)
3
 We can always split the problem into at least 2/3 the size. Consider the
number of nodes on the left side vs. the right in the most unbalanced
state:
 In the worst case a heap of height 𝑛 has all of the bottom leaves of the
left child filled and the right child has height 𝑛 − 1. This is the most
unbalanced a tree will ever become due to the heap property.
- Complete binary tree: 2k leaves, 2k-1 interior nodes
14/30
 Building a heap
 Given an array 𝐴[1 … . . 𝑛], we want to build this array into a max-heap
using the procedure Max-Heapify
 Note: Leaves “𝐴[( 𝑛/2 + 1) … . 𝑛]” are already a heap! So start to build
from the leaves and continue recursively to call Max-Heapify, moving up the
tree to the root.

15/30
 Example of Building a heap
 Build-Max-Heap(A) 1
1
A=[1 5 9 4 7 10 2 6 3 14]
2 3
Max-Heapify(A,10) exits since this is a leaf.
Max-Heapify(A,9) exits since this is a leaf. 5 9
Max-Heapify(A,8) exits since this is a leaf. 6
4 5 7
Max-Heapify(A,7) exits since this is a leaf.
Max-Heapify(A,6) exits since this is a leaf. 4 7 10 2

8 9 10

6 3 14
16/30
 Example of Building a heap
1. Max-Heapify(A,5) puts the largest of A[5] and its children, A[10] into A[5]
1 1
1 1

2 3 2 3
5 9 5 9
4 5 6 7 4 5 6 7

4 7 10 2 4 14 10 2

9 10 8 9 10
8

6 3 14 6 3 7

A=[1 5 9 4 7 10 2 6 3 14] A=[1 5 9 4 14 10 2 6 3 7] 17/30

 Example of Building a heap
2. Max-Heapify(A,4)
1
1 1
1
2 3
2 3
5 9
5 9
4 5 6 7
4 5 6 7
4 14 10 2 6 14 10 2
8 9 10
8 9 10
6 3 7 4 3 7
A=[1 5 9 4 14 10 2 6 3 7] A=[1 5 9 6 14 10 2 4 3 7] 18/30
 Example of Building a heap
3. Max-Heapify(A,3)
1 1
1 1

2 3 2 3
5 9 5 10
4 5 6 7 4 5 6 7
6 14 10 2 6 14 9 2
8 9 10
8 9 10
4 3 7
4 3 7
A=[1 5 9 6 14 10 2 4 3 7] A=[1 5 10 6 14 9 2 4 3 7] 19/30
 Example of Building a heap
4. Max-Heapify(A,2) 5. Max-Heapify(A,5)
1
1 1
1
1 1
2 3
5 10 2 3 2 3
14 10 14 10
4 5 6 7
4 5 6 7 4 5 6 7
6 14 9 2
6 5 9 2 6 7 9 2
8 9 10
9 10 8 9 10
8
4 3 7
4 3 7 4 3 5

A=[1 5 10 6 14 9 2 4 3 7] A=[1 14 10 6 5 9 2 4 3 7] A=[1 14 10 6 7 9 2 4 3 5]

20/30
 Example of Building a heap
6. Max-Heapify(A,1) 7. Max-Heapify(A,2)
1
1 1
14
2 3
14 10 2 3
1 10
4 5 6 7
4 5 6 7
6 7 9 2
6 7 9 2
8 9 10
8 9 10
4 3 5
4 3 5
A=[1 14 10 6 7 9 2 4 3 5] A=[14 1 10 6 7 9 2 4 3 5] 21/30
 Example of Building a heap
1 1
14 8-Max-Heapify(A,5) 14

2 3 2 3
7 10 7 10
4 5 6 7 4 5 6 7

6 1 9 2 6 5 9 2
8 9 10 8 9 10

4 3 5 4 3 1

A=[14 7 10 6 1 9 2 4 3 5] A=[14 7 10 6 5 9 2 4 3 1]

 Finished Max-Heap: A=[14 7 10 6 5 9 2 4 3 1] 22/30

 Build-Heap Runtime
• Running Time
– We have a loop of 𝑛 times, and each time call Max-Heapify
which runs in (𝑙𝑔𝑛). This implies a bound of 𝑂(𝑛𝑙𝑔𝑛). This
is correct, but is a loose bound! We can do better.

– Key observation: Each time Max-Heapify is run within the

loop, it is not run on the entire tree. We run it on subtrees,
which have a lower height, so these subtrees do not take 𝒍𝒈𝒏
time to run. Since the tree has more nodes on the leaf, most
of the time the heaps are small compared to the size of 𝑛.
23/30
 Build-Heap Runtime
• Property: In an 𝑛-element heap there are at most
𝑛/(2ℎ) nodes at height ℎ
– The leaves are ℎ=1 and root at 𝑙𝑔𝑛, this is a
slight change from the previous definition
(leaves at height 0)
• The time required by Max-Heapify when called in
Build-Max-Heap on a node at height ℎ is 𝑂(ℎ);
ℎ=𝑙𝑔𝑛 for the entire tree.
24/30
 Build-Heap Runtime
• Cost of Build-Max-Heap is then:
heap _ height
T ( n)   (# nodes _ at _ h)(Heapify  Time)
h 1
lg n
n
T ( n)   h
O ( h)
h 1 2
 lg n n 
T ( n)  O  h h 
 h 1 2 

25/30
 Heapsort Algorithm
 Once we can build a Max-Heap and Max-Heapify, sorting is easy

 The heapsort algorithm starts by using BUILD-MAX-HEAP to build a max-Heap on

the input array 𝐴[1 … 𝑛]. Since the maximum element of the array is stored at the
root 𝐴[1], we can put it into its correct final position by exchanging it with 𝐴[𝑛].

• If we now discard node 𝑛 from the heap—and we can do so by simply

decrementing 𝐴. 𝐻𝑒𝑎𝑝-𝑆𝑖𝑧𝑒 —we observe that the children of the root remain Max-
Heaps, but the new root element might violate the Max-Heap property.

• All we need to do to restore the Max-Heap property, however, is call Max-

Heapify(𝐴,1), which leaves a Max-Heap in 𝐴[1 … 𝑛 − 1].
26/30
 Heapsort Algorithm(Cont.)
 The heapsort algorithm then repeats this process for the Max-Heap of size 𝑛 − 1
down to a heap of size 2

 Runtime is 𝑂(𝑛𝑙𝑔𝑛) since we do Max-Heapify on 𝑛 − 1 elements, and we do

Max-Heapify on the whole tree.
 Note: In-place sort, required no extra storage variables unlike Merge Sort, which
used extra space in the recursion. 27/30
 Example of Heapsort
 An example of the operation of Heapsort after line 1 has built the initial
Max-Heap
[16,14,10,8,7,9,3,2,4,1] [14,8,10,4,7,9,3,2,1]--- [16]

28/30
 Example of Heapsort(cont.)
[10,8,9,4,7,1,3,2]– [14,16] [9,8,3,4,7,1,2] – [10,14,16]

[8,7,3,4,2,1] – [9,10,14,16] [7,4,3,1,2] – [8,9,10,14,16]

29/30
 Example of Heapsort(cont.)
[4,2,3,1] – [7,8,9,10,14,16] [3,2,1] – [4,7,8,9,10,14,16] [2,1] – [3,4,7,8,9,10,14,16]

[1] –[2,3,4,7,8,9,10,14,16]
Sorted Array

30/30