ESc101 : Fundamental of Computing

I Semester 2008-09

Lecture 37

• Analyzing the efficiency of algorithms.

• Algorithms compared
– Sequential Search and Binary search

– GCD fast and GCD slow

– Merge Sort and Selection Sort

1
Problem 1 : Searching

2
 Comparing Searching algorithms

Searching for an element x in a sorted array of n numbers.

• Sequential Search
• Binary search
Experimental observation : Binary search has been found to be much faster
than sequential search. (given as assignment during Lab 11 for Thursday and
Friday)

3
Problem 2 : Computing GCD of two positive numbers

4
 Comparing Two GCD algorithms

We gave two algorithms for GCD computation long back in this course.

• GCD fast : based on %

• GCD slow : based on subtraction

The code of these algorithms is shown on the following page.

5
 Why is GCD fast faster than GCD slow ?
//Assume m>=n
int GCD_fast(m,n)
{ while(n!=0)
{ rem = m%n;
m = n;
n = rem;
}
return m;
}
--------------------------------
int GCD_slow(m,n)
{ while(n!=0)
{ diff = m-n;
if(diff>=n) m = diff
else {m=n; n = diff;}
}
return m;
}

6
 Problem 3 : Sorting

Given an array storing n numbers, sort them

7
 Comparing Three sorting algorithms

Experimental Observations

• Quick sort is more efficient than merge sort

• Merge sort is more efficient than Selection Sort

The code is given in Three sorting algos.java

8
 What is the reason for different running times?

Given that all algorithms (for searching, GCD, sorting)

• have same input and output

• are executed in same environment (machine, operating system)

We need to analyze the number of steps/instruction

taken by each algorithm for a problem

9
Algorithm design is incomplete until you analyze its running time

10
How many steps/instructions are executed by the following loop ?

for(int i=1; i<=n; i=i+1)

{
sum = sum + i;
}

For sake of simplicity, we can say that

No. of Steps = 1+3n+1

11
How many steps/instructions are executed by the following loop ?

for(int i=1; i<=n; i=i+1)

{
sum = sum + i;
}

For sake of simplicity, we can say that

No. of Steps = 1 + 3n + 1

12
How many steps/instructions are executed by the following loop ?

for(int i=1; i<=n; i=i+1)

{
sum = sum + i;
}

For sake of simplicity, we can say that

No. of Steps = an + b, for some positive constants a, b

13
How many steps/instructions are executed by the following loop

for(int n=1;n<=m;n=n+1)
{
for(int i=1; i<=n; i=i+1)
{
sum = sum + i;
}
}

For sake of simplicity, we can say that

Pn=m
No. of Steps = 1 + m + n=1 (1 + 3n + 1) + m

14
How many steps/instructions are executed by the following loop

for(int n=1;n<=m;n=n+1)
{
for(int i=1; i<=n; i=i+1)
{
sum = sum + i;
}
}

For sake of simplicity, we can say that

Pn=m
No. of Steps = 1 + m + n=1 (1 + 3n + 1) + m = 3/2m2 + 11/2m + 1

15
How many steps/instructions are executed by the following loop

for(int n=1;n<=m;n=n+1)
{
for(int i=1; i<=n; i=i+1)
{
sum = sum + i;
}
}
For sake of simplicity, we can say that
No. of Steps = am2 + bm + c, for some constants a,b,c

16
 Analysis of Number of instructions of an algorithm

How many instructions are executed ...

• to search a number in an unsorted array storing n numbers.

• to search a number in a sorted array of m numbers.
• to sort n numbers by selection sort.
• to sort n numbers by merge sort.
it is function of imput size

We shall focus on worst case number of instructions taken by an algorithm

17
 Analysis of Number of instructions of an algorithm

How many instructions are executed ...

• to search a number in an unsorted array storing n numbers.

• to search a number in a sorted array of m numbers.
• to sort n numbers by selection sort.
• to sort n numbers by merge sort.
Observation : it is function of input size

We shall focus on worst case number of instructions taken by an algorithm

18
No. of Instructions executed during Sequential Search on n numbers

• For sequential search, you can write a for loop for the sequential search
which iterates n times in the worst case.

• In each iteration, we perform constant number of instructions

No. of instructions in the worst case :

cn for some constant c

19
 No. of Instructions executed during Binary Search
Given that array A is sorted.

public static boolean Bin_search(int[] A, int x)

{ int left =0;
int right=A.length-1;
boolean Is_found = false; int mid;

while(Is_found==false && left>right)

{
mid = (left+right);
if(A[mid]==x) Is_found=true;
else if(A[mid]>x) right = mid-1;
else left = mid+1;
}
return Is_found;
}

20
 Analysis of Binary Search

• There are four instructions before entering the while loop.

• Number of instructions in each iteration of while loop is at most 5.
• After each iterations of the while loop, the search domain (A[left]..A[right])
reduces by at least a factor of 2

• Total number of iterations of loop : log2 n.

Hence the number of instructions in the worst case =

4 + 5 log2 n = a log2 n + b, for some constants a, b.

21
Hence Binary Search is exponentially faster than sequential search

22
 Why is GCD fast faster than GCD slow ?
//Assume m>=n
int GCD_fast(m,n)
{ while(n!=0)
{ rem = m%n;
m = n;
n = rem;
}
return m;
}
--------------------------------
int GCD_slow(m,n)
{ while(n!=0)
{ diff = m-n;
if(diff>=n) m = diff
else {m=n; n = diff;}
}
return m;
}

23
 Analysis of GCD fast
We shall bound the number of iterations of the while loop.

//Assume m>=n
int GCD_fast(m,n)
{ while(n!=0)
{ rem = m%n;
m = n;
n = rem;
}
return m;
}

Observation : After an iteration (m, n) −→ (n, m%n).

24
 Analysis of GCD fast

Observation : After an iteration (m, n) −→ (n, m%n).

Consider any iteration.

• Case 1 : m > 32 n
Inference : after the iteration
the new value of m < 23 times old value of m

• Case 2 : m ≤ 32 n
Inference : after the iteration
The new value of n ≤ 12 times old value of n

25
 Analysis of GCD fast

Observation : After an iteration (m, n) −→ (n, m%n).

Consider any iteration.

• Case 1 : m > 32 n
Inference : after the iteration
2
the new value of m < 3 times old value of m

• Case 2 : m ≤ 32 n
Inference : after the iteration
The new value of n ≤ 12 times old value of n

26
 Analysis of GCD fast

Observation : After an iteration (m, n) −→ (n, m%n).

Consider any iteration.

• Case 1 : m > 32 n
Inference : after the iteration
2
the new value of m < 3 times old value of m

• Case 2 : m ≤ 32 n
Inference : after the iteration
The new value of n ≤ 1
2 times old value of n

27
 Analysis of GCD fast

• It can be seen that once m or n becomes less than or equal to 2, at most one
more iteration will be executed.

• Hence, based on previous slide the number of iterations of while loop is at

most log3/2 m + log2 n.

28
 Analysis of GCD slow
//Assume m>=n
int GCD_slow(m,n)
{ while(n!=0)
{ diff = m-n;
if(diff>=n) m = diff
else {m=n; n = diff;}
}
return m;
}

m is much larger than n.

The worst case :
For example m = 10000000002, n = 2.
It follows from the code that the algorithm will perform m/n iterations which is
close to m when n is a small number.

29
 Comparing GCD fast and GCD slow

In the worst case,

GCD fast is exponentially faster than GCD slow.

30
Comparing Selection Sort and Merge Sort

31
No. of instructions executed in Selection Sort on n numbers

32
 Number of instructions taken in Selection Sort
int index_of_smallest_value(int[] A,int i)
//returns integer j such that A[j] is smallest among A[i], A[i+1],...

SelectionSort(int [] A)
{

for(int count=0;count<A.length;count=count+1)
{
int j = index_of_smallest_value(A, count);
if(j != count)
swap_values_at(A,j,count);
}
}

It follows easily that index of smallest value takes c(n − i)

instructions in the worst case for some constant c.

33
Number of instructions taken in Selection Sort on n numbers

• There are ???? iterations of the for loop

• Number of instructions taken to find ith smallest element = ????

34
Number of instructions taken in Selection Sort on n numbers

• There are n − 1 iterations of the for loop

• Number of instructions taken to find ith smallest element = ???? c(n − i) for
some constant c.

35
Number of instructions taken in Selection Sort on n numbers

• There are n − 1 iterations of the for loop

• Number of instructions taken to find ith smallest element = c(n − i) for some
constant c.

36
Number of instructions taken in Selection Sort on n numbers

X n(n − 1)
c(n − i) = c
2
0≤i<n−1

37
Let us analyse Merge Sort on n numbers

It uses a method to merge two sorted arrays

38
number of instructions for merging two sorted arrays of size n

Recall that the algorithm proceeds like :

start scanning A and B from left, compare two elements of A and B , copy the
smaller one into C and continue ...

39
 Merging two sorted arrays

A 5 9 28 49 89 101 ...

B 3 34 40 53 66 92 ...

C 3 5 9 28 34

Number of instructions : cn for some constant c

40
 Number of instructions taken in Merge Sort
public static void mergesort(int[] A, int left, int right)
{ if(left!=right)
{ int mid = (left+right)/2;
mergesort(A, left, mid);
mergesort(A, mid+1, right);
merge(A,left,mid,right);
}
}

Each call of mergesort does two tasks

• Invoking two calls recursively

• Merging of two sorted portions of array A

41
 Number of instructions taken in Merge Sort
MSort(A,0,7)

Level 3
MSort(A,4,7)
MSort(A,0,3)

Level 2

MSort(A,0,1) MSort(A,2,3) MSort(A,4,5) MSort(A,6,7)

Level 1

Level 0
MSort(A,0,0) MSort(A,1,1) MSort(A,2,2) MSort(A,3,3) MSort(A,4,4) MSort(A,5,5) MSort(A,6,6) MSort(A,7,7)

A 99 7 5 1 67 11 4 2

42
 Number of instructions taken in Merge Sort

• We perform cn instructions at each level of the recursion tree.

• There are log2 n levels in the tree.
Hence in the worst case there are cn log2 n instructions executed in Merge Sort
on an array of size n.

43
 Alternate analysis of Merge Sort

Let T (n) be the number of instructions executed by merge sort on n numbers.

The following recurrence captures the running time of merge sort exactly.

 a if n =1
T (n) =
 cn + 2T (n/2) otherwise

Here cn is the no. of instructions for merging two halves of the array.

44
 Alternate analysis of Merge Sort

Gradually unfold the recurrence.

T (n) = cn + 2(cn/2 + 2T (n/22 ))

= cn + cn + 22 T (n/22 )
= cn + cn + cn + ... + 2i T (n/2i )
= cn + cn + cn + ... log2 n terms...
= cn log2 n

45
Efficiency of Quick Sort to be analysed in next lecture class

Please come to Wednesday lecture with any question/doubt

46