Introduction

What is an Algorithm?
The word “algorithm” came from the name of a Persian Mathematician Abu Ja’far
Mohammed ibn Musa al Khowarizmi.

Definition: An algorithm is a finite set of steps or instructions to accomplish a

particular task or to solve a particular problem written in a convenient language.

All algorithms should satisfy the following five criteria:

1> Input: Zero or more quantities are externally supplied, i.e., there may be some
algorithms without any externally supplied input.
2> Output: At least one quantity is produced as an outcome, i.e., every algorithm
has to produce some output.
3> Definiteness: Every step in the algorithm should be clear, i.e., without
ambiguity.
The following steps are ambiguous :
Subtract 2 or 3 from y.
Division by zero.

To achieve this, the algorithms are expressed using a programming language as a

program.
The programming languages are designed such that each statement and/or
sentence has a unique meaning.
4> Finiteness: Under all circumstances, the algorithm should terminate after a
finite number of steps. The time taken for termination should be reasonably short.
5> Effectiveness: Every instruction in the algorithm must be basic so that it can
be carried out using pen and paper by us.
For example, arithmetic operations on integers is an effective operation, but on real
numbers is not.

Computational Procedures are the algorithms that are definite and effective.
Popular example for this is OS, which does not terminate, and waits for the next
job.

There are four distinct areas of study for the algorithms :

1> How to device algorithms?
Writing an algorithm is an art and it may never be automated. There are various
useful design techniques, which yield good algorithms.
By learning and understanding these techniques, new and useful algorithms can be
devised.
All of these techniques are applicable in various fields of study such as operations
research, electrical engineering including computer science.

2> How to validate algorithms?

After devising the algorithm, it is necessary to check, whether it produces correct
answer for all possible types of inputs.
This process is known as algorithm validation.
 For algorithm validation, it need not be expressed as a program. So, the algorithm
validation ensures that it works correctly independent of the programming
language.

After that it will be expressed as a program and program proving or program

verification is carried out.

For proof of correctness, the solution has to be stated in two forms:

1> First form is as a program, annotated by a set of assertions about the input and
output variables, expressed in the predicate calculus.
2> Second form is called as specification, also expressed in the predicate calculus.

Now, the proof consists of showing, for every legal input, these two forms are
equivalent.

For a complete proof of correctness:

- Each statement of the program is precisely defined.
- All basic operations are proved.

3> How to analyze algorithms?

When an algorithm is executed, it uses:
- The Central Processing Unit (CPU) of the computer to perform operations
(Time).
- The memory of the computer to hold both the program and data (Space).
The process of determining the amount of computing time and storage space
required by an algorithm is known as Analysis of algorithms or Performance
Analysis.

This analysis results in:

- Quantitative judgments about the value of one algorithm over another.
- Predicting whether the software meets existing efficiency constraints.
- Evaluation of algorithm’s performance in the best, worst and average cases.

4> How to test a program?

The program testing consists of two phases:
- Debugging
- Profiling (Performance Measurement)

Debugging: It is the process of executing programs on sample data sets to

determine whether faulty results occur, if so, correct them.
So, it only points the presence of errors, but not their absence.

A proof of correctness is much valuable, because it guarantees that the program

works correctly for all possible inputs.

Profiling: It is the process of executing the correct program on data sets and
measuring the time and space taken by it to get the results.
This helps in determining the efficiency of the program.
 Algorithm Specification
Pseudocode Conventions:
An algorithm can be expressed or described in many ways, the following two are
popular :
1> Using natural language like English – the resulting instructions may be
ambiguous i.e., not definite.
2> Using the graphic representations, flowcharts – suitable if the algorithm is small
and simple.

So, a pseudocode, that resembles the syntax of Pascal, C and/or C++ languages is
required to express algorithms. The conventions used are :
1> Comments begin with // and continue until the end of line.
2> Simple statement is delimited by ;.
A collection of simple statements, called as a compound statement, can be
represented as block.
The body of a procedure is also represented as a block.
Blocks are indicated with matching braces { and }.
Ex:
{ simple statement 1;
simple statement 2; …
simple statement n;
}
3> An identifier or a variable-name begins with a letter.
From the context, the data type and scope of the variable is evident. So, they are
not explicitly specified.
The assumed simple data types are integer, float, char, boolean and so on.
The compound data types can be formed with records, as shown below:
node = record
{ datatype _1 data_1;
.
.
.
datatype_n data_n;
node *link;
}
Individual data items of a record can be accessed with  and . (period).

For example, if ‘p’ is a pointer to a record type node, then p  data_1 stands for
the value of the first field. If ‘q’ is a record of type node, then q.data_1 denote the
value of the first field

4> Assignment of values to variables is done using the assignment statement.

variable := expression;
Ex: a := 5;
cap := x + y * z;
5> Two boolean values true and false are available.
To produce these values, the following operators are available :
- Logical operators and, or and not.
- Relational operators <, ≤, =, ≠, ≥ and >.

6> Array indices start at zero.

The elements of the arrays are accessed using [ and ].
Ex: ith element of a one dimensional array ‘A’ is denoted as A[i]. Similarly, (i, j) th
element of a two dimensional array ‘B’ is denoted as B[i, j].

7> The looping statements for, while and repeat-until are used to repeat
statement(s).
while: The while loop is of the following form:
while condition do while (j < 10) do
{ statement 1; {
. sum := sum + j;
. j := j + 1;
. }
statement n;
}
The statement can be simple or compound.
The value of ‘condition’ is evaluated at the top of the loop, if it true, then the loop
gets executed, otherwise, the loop is exited.
for: The for loop is of the following form:
for variable := value1 to value2 step modifier do for i:= 1 to 10 step 2 do
{ statement 1; {
. sum := sum + i;
. }
.
statement n;
}
The value1, value2 and modifier are arithmetic expressions (constants also
included).
The condition, (variable ≤ value2) is tested at the beginning of each iteration, if it is
true, the loop continues, otherwise it is exited.
The clause ‘step modifier’ is optional. If not there, an increment of +1 is assumed.
The modifier can be either positive or negative.

The for loop can be implemented as while loop as shown below:

var1 := value1; var2 := value2; incr := modifier;
while ((var1 – var2) ≤ 0) do
{ statement 1;
.
.
.
statement n;
var1 := var1 + incr;
}
repeat-until: The syntax for this loop is as given below:
repeat repeat
statement 1; sum := sum + j;
. j := j + 1;
. until (j > 10);
.
statement n;
until condition;
The statements enclosed between repeat-until are executed as long as condition is
false. In this, the condition is computed after executing the statements.

The break instruction can be used within the loop statements, to force exit. In the
case of nested loops, there will be an exit from the innermost loop.
The return statement results in, not only exiting from the loops and also from the
function itself.

8> A conditional statement has the following forms :

if condition then statement;
if condition then statement 1; else statement 2;
Here, condition is a boolean expression and statement, statement 1 &
statement 2 can be either simple or compound.

The then and/or else part of an if statement can have another if statement. This
case is called as nested if statement. This is used for multiple condition checking.
For multiple condition checking, case statement is used and its form is:
case
{
: condition 1 : statement 1;
: condition 2 : statement 2;
.
.
.
: condition n : statement n;
: else : statement n+1;
}

Here, statement 1, …, statement n+1 can be either simple or compound.

The case statement is executed as follows:
- If condition 1 is true, the statement 1 gets executed and the case statement
is exited, otherwise condition 2 is evaluated.
- If condition 2 is true, the statement 2 gets executed and the case statement
is exited, otherwise the next case is evaluated and so on.
- If none of the specified conditions are true, then the statement n+1 is
executed and the case statement is exited. The else clause is optional.

9> The read and write instructions are used for input and output operations
respectively. No format is used to specify the type and size of input and output
quantities.
10> Algorithm is the only type of procedure. An algorithm consists of two parts,
namely header and body.

The form of the header is:

Algorithm Name (parameter list)
Here, Name is the procedure name and parameter list is the formal parameters’
list of the procedure.
The body has one or more simple and/or compound statements enclosed within the
braces { and }.

Simple variables are passed by value and, the arrays & records are passed by
reference i.e., as a pointer to the respective data type.
An algorithm may or may not return any value.

Ex : The algorithm for finding and returning, the maximum of n given numbers.
Algorithm Max (A, n)
{
Maxi := A[1];
for i:= 2 to n do
if A[i] > Maxi then Maxi := A[i];
return Maxi;
}
Here, A and n are formal parameters. Maxi and i are local variables.
 Selection Sort Example
(With Minimum Element)
8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9
Selection Sort Example (With Maximum Element)
Before sorting 14 2 10 5 1 3 17 7

After pass 1 14 2 10 5 1 3 7 17

After pass 2 7 2 10 5 1 3 14 17

After pass 3 7 2 3 5 1 10 14 17

After pass 4 1 2 3 5 7 10 14 17
Ex: Selection Sort – An algorithm for sorting a collection of n ≥ 1 elements of
arbitrary type.
A simple solution can be stated as:
From those elements that are currently unsorted, find the smallest and
place it next in the sorted list.
Even though, the above statement adequately describes the sorting, it is not an
algorithm and leaves the following questions unanswered.

1> Where the elements are initially stored?

2> How the elements are stored?
3> Where to place the result?

Now, let us assume that the elements are stored in an array a, such that the ith
element is stored at a[i], 1 ≤ i ≤ n.
With the above assumptions the modified algorithm is:
for i := 1 to n do
{
Examine a[i] to a[n] to find the smallest element, let it be at a[j];
Interchange a[i] and a[j];
}
The above algorithm, involves two subtasks:
1> Finding the smallest element (say a[j]).
2> Interchanging it with a[i].
For the first subtask :
- Initially assume a[i] as minimum.
- Now compare it with a[i+1], a[i+2], …, whenever smaller element is found,
make it new minimum.
- The above process is continued till a[n] is compared. Let the minimum element
is a[j].

For the second subtask, the following code is used.

t := a[i];
a[i] := a[j];
a[j] := t;
Now, the modified algorithm for selection sort is :
Algorithm SelectionSort(a, n) Ex : i [0] [1] [2] [3] [4]
{ - 30 10 50 40 20
for i := 1 to n - 1 do 1 10 30 50 40 20
{ 2 10 20 50 40 30
j :=i; 3 10 20 30 40 50
for k := i + 1 to n do 4 10 20 30 40 50
if a[k] < a[j] then j := k;
t := a[i]; a[i] := a[j]; a[j] := t;
}
}
Note : The notation a[i : j] denotes array elements a[i] through a[j]
 Insertion Sort
Idea: like sorting a hand of playing cards
– Start with an empty left hand and the cards facing
down on the table.

– Remove one card at a time from the table, and insert

it into the correct position in the left hand
• compare it with each of the cards already in the hand, from
right to left

– The cards held in the left hand are sorted

• these cards were originally the top cards of the pile on the
table
 To insert 12, we need to make
room for it by moving first 36
and then 24.
6 1 0 2 4 36

12
6 10 24 36

12
6 10 24 3
6

12
 Example
input array

5 2 4 6 1 3

at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted
 Bubble Sort
o Compare each element (except the last one) with its
neighbor to the right
 If they are out of order, swap them
 This puts the largest element at the very end
 The last element is now in the correct and final place

o Compare each element (except the last two) with its

neighbor to the right
 If they are out of order, swap them
 This puts the second largest element next to last
 The last two elements are now in their correct and final places

o Compare each element (except the last three) with its

neighbor to the right
 Continue as above until no unsorted elements are present on
the left
 Example

7 2 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8

2 7 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8

2 7 8 5 4 2 5 7 4 8 2 4 5 7 8 (done)

2 7 5 8 4 2 5 4 7 8

2 7 5 4 8
Recursive Algorithms:
There are two types of recursion:
1> Direct Recursion
2> Indirect Recursion

An algorithm is said to be direct recursive if the same algorithm is invoked in the

body.
Ex: Algorithm a(n)
{
---
a(n – 1);
---
}
An algorithm ‘A’ is said to be indirect recursive if it calls another algorithm, which
in turn calls A. Both A and B are called as Co-routines.
Ex: Algorithm a(n) Algorithm b(m)
{ {
--- ---
b(n – 1); a(m – 1);
--- ---
} }
Recursive mechanism is extremely powerful and complex. For some people it is
mystical technique.
But, any algorithm that can be written using
- An assignment,
- The if-then-else statement &
- The loop statement
can also be written using
- An assignment,
- The if-then-else statement &
- The recursion.
Recursion will be appropriate mechanism for algorithm design, if the problem itself
is recursively defined. Ex. Factorials and binomial coefficients.

The recursive algorithm for factorial is :

Algorithm f(n)
{
if n ≤ 1
return 1;
else
return n * f(n – 1);
}
 Example

Compute 5!

L16 27
 f(5)=
5·f(4)

L16 28
 f(4)= f(5)=
4·f(3) 5·f(4)

L16 29
 f(3)= f(4)= f(5)=
3·f(2) 4·f(3) 5·f(4)

L16 30
 f(2)= f(3)= f(4)= f(5)=
2·f(1) 3·f(2) 4·f(3) 5·f(4)

L16 31
 f(1)= f(2)= f(3)= f(4)= f(5)=
1 2·f(1) 3·f(2) 4·f(3) 5·f(4)

L16 32
 2·1= f(3)= f(4)= f(5)=
2 3·f(2) 4·f(3) 5·f(4)

L16 33
 3·2= f(4)= f(5)=
6 4·f(3) 5·f(4)

L16 34
 4·6= f(5)=
24 5·f(4)

L16 35
 5·24=
120

L16 36
 Return 5!
= 120

L16 37
 Fibonacci number w/o
recursion
Algorithm fib(n)
{
f[0] := 0; f[1] := 1;
for i := 2 to n
f[i] := f[i-1] + f[i-2];
return f[n];
}
Fibonacci Numbers
Now, to develop recursive algorithms, let us consider the following two problems :
1> Towers of Hanoi problem
2> Permutation Generator problem – generates possible permutations for a list of
elements.

1> Towers of Hanoi Problem : This problem is based on the ancient Tower of
Brahma ritual.

According to this :
- When the world was created, there was a diamond tower (A) with 64 golden disks.
- The disks were placed on the tower in decreasing order of size from bottom to top.
- Besides this tower, there were two other diamond towers (B & C).
- From the creation of world, the Brahman priests have been attempting to move the
disks from A to B using C for intermediate storage.
- Since the disks are very heavy, they can be moved only one at a time.
- At any time, a disk can not be on the top of a smaller disk.
- According to legend, the world will come to an end, if the priests complete the task.
 The efficient algorithm for this problem can be obtained using the recursion.
- Assume that there are n disks, to get the largest disk to the bottom of B, the
remaining n-1 disks are moved to C.
- Now, the remaining disks have to be moved from C to B, using A and B.

The recursive algorithm for this problem is :

Algorithm TowersOfHanoi (n, x, y, z)

// Move the n disks from tower x to tower y
{
if (n ≥ 1) then
{
TowersofHanoi (n-1, x, z, y);
write (“move top disk from tower” ,x ,”to
top of tower”, y);
TowersofHanoi (n-1, z, y, x);
}
}

Observation : Here, the solution to n-disk problem is formulated in terms of two

(n-1)-disk problems.

In this algorithm, the tower A is denoted by x, B by y and C by z. n is the number

of disks.
2> Permutation Generator : Given a set of n ≥ 1 elements, the problem is to print
all possible permutations of this set.
Ex : The set of permutations for a given set of elements {a, b, c} is {(a, b, c), (a, c,
b), (b, a, c), (b, c, a), (c, a, b), (c, b, a)}.

For a given set of n elements, there will n! different permutations. This problem can
be defined recursively.
Let us take a set of four elements {a, b, c, d}. The solution can be defined as :
1> a followed by all the permutations of (b, c, d).
2> b followed by all the permutations of (a, c, d).
3> c followed by all the permutations of (a, b, d).
4> d followed by all the permutations of (a, b, c).
The recursive algorithm for this problem is :
Algorithm Perm (a, k, n)
{ if (k = n) then write (a[1:n]);
else
for j := k to n do
{
t := a[k]; a[k] := a[j]; a[j] := t;
Perm(a, k + 1, n);
t := a[k]; a[k] := a[j]; a[j] := t;
}
}
 Performance Analysis
An algorithm evaluation can be based upon many criteria, some of them are :
1> Does it do what we want it to do?
2> Does it work correctly according to the original specifications of the task / problem?
3> Is there documentation that describes how to use it and how it works?
4> Are procedures created in such a way that they perform logical sub-functions?
5> Is the code readable?

The above criteria are very important in writing software, especially for large
systems. These criteria are automatically met in our algorithms.
The other criteria, which have a more direct relationship to evaluate the
performance of algorithms are :
- Computing time
- Storage requirements.

Space Complexity : The space complexity of an algorithm is the amount of

memory required to complete the execution.

Time Complexity : The time complexity of an algorithm is the amount of

processor (CPU) time required to complete the execution.
 The performance evaluation can be divided into two major phases :
1> A Priori estimates (Performance Analysis)
2> A Posteriori testing (Performance Measurement)

Space Complexity :
The space needed by an algorithm, is the sum of the following two components :
1> A fixed part, independent of the characteristics such as the size and number of
inputs and outputs. It includes :
- The instruction space
- Space for simple variables
- Space for fixed-size compound variables
- Space for constants and so on.

2> A variable part, is the space needed by component variables, whose size is
dependent on the instance of a problem being solved. It includes :
- The space needed by the referenced variables
- The recursion stack space.

 The space requirement S(P) of any algorithm P is expressed as :

S(P) = c + SP (instance characteristics)
where c is a constant and represents fixed part
SP (instance characteristics) represents variable part
 To analyze the space complexity of an algorithm, the estimation of S P (instance
characteristics) is crucial and important.
For this, the instance characteristics required to measure the space requirements
have to be determined.
In many cases, it is limited to the quantities related to the number and magnitude
of the inputs and outputs.
But, in some cases, more complex measures such as the interrelationships among
the data items also used.

Ex : Write an algorithm to compute

a + b + b * c + (a + b – c) / (a + b) + 4.0
and estimate the space requirements.

Algorithm abc (a, b, c)

{
return a + b + b * c + (a + b – c) / (a + b) + 4.0;
}

This algorithm instance uses the specific values of a, b, and c. If one word is
required for each variable, the total space needed is ≤ 5.

This instance is independent of the instance characteristics.

Sabc (instance characteristics) = 0
 Algorithm Sum(a, n)
{
s := 0.0;
for j := 1 to n do
s := s + a[j];
return s;
}

The problem instances for this algorithm are characterized by the

value of n, the number of elements to be summed.

For this algorithm :

- One word each is required for n, j, and s.
- The space needed by a is at least n words.

 SSum (n) = n.
 S(Sum) ≥ (n + 3).
Ex : Write an algorithm, that computes

recursively, where a[i]s are real numbers. Also estimate the space
requirements.
Algorithm SumR(a, n)
{
if (n  0) then return 0.0;
else return SumR(a, n-1) + a[n];
}
The problem instances for this algorithm are characterized by :
- The value of n
- The recursion stack space, consists the space for :
 Formal parameters
 Local variables
 Return address (Let us assume one word)

So, each call to SumR requires at least 3 words :

- The value of n.
- The return address.
- The pointer to a[].

The depth of the recursion is n+1.

 The recursion stack space required is ≥ 3(n + 1)
Time Complexity :
Actually, the time T(P) taken by a program is the sum of :
- The compile time.
- The run or execution time.
The compile time does not depend on the instance characteristics.

Since the compiled program can be executed many times, only the execution time
is considered and it is denoted by tP (instance characteristics).

The value of tP depends on many factors such as the numbers of additions,

subtractions, multiplications, divisions, compares, loads, stores and so on.

So, only the estimation for tP can be made and the expression is :
tP (n) = Ca ADD(n) + Cs SUB(n) + Cm MUL(n) + Cd DIV(n) + …
where
- n denotes the instance characteristics.
- Ca, Cs, Cm, Cd, and so on, denote time needed for an addition, subtraction,
multiplication, division, and so on respectively.
- ADD, SUB, MUL, DIV, and so on are functions.

Here, the time needed for addition, subtraction, multiplication, division, so on

depends on the numbers (size and type).
The value tP (n) in
- In single-user system depends on the type of the processor.
- In multi-user system depends on
 System load
 The number of other programs running
 The characteristics of these programs.
So, tP (n) is the estimation of only the number of program / algorithm steps.
A program step is defined as a syntactically or semantically meaningful segment of
a program. Its execution time is independent of instance characteristics.
Ex : return a + b + b * c + (a + b – c) / (a + b) + 4.0; is taken as a single step.
The number of steps assigned to a program statement depends on its kind. For
example
- Comments count as zero steps.
- Assignment statement without the calls is counted as one step.
- In iterative statements, the step counts are considered only for the control part.

The number of steps needed by a program, to solve a particular problem instance,

can be determined in one of the two ways.
First Method :
- Introduce a new global variable count, with initial value 0;
- Statements to increment count by an appropriate amount are introduced.
So, now the count is incremented by the step count of each statement, which is
executed.
Ex : Let us consider the algorithm Sum and introduce the global variable count into it.
Algorithm Sum(a, n)
{
s := 0.0;
count := count + 1; // for assignment of s
for j := 0 to n-1 do
{
count := count + 1; //for j
s := s + a[j];
count := count + 1; //for s
}
count := count + 1; // for last iteration of j
count := count + 1; // for return
return s;
}

The above algorithm can be simplified with only the count increments, as shown
below:
Algorithm Sum(a, n)
{
for j := 0 to n-1 do
count := count + 2;
count := count + 3;
}
Ex : Now let us consider the algorithm SumR, which recursively finds the sum of n
numbers, stored in an array.
Algorithm SumR(a, n)
{ count := count + 1;
if (n ≤ 0) then
{ count := count + 1;
return 0.0;
}
else
{count := count + 1;
return SumR(a, n-1) + a[n];
}
}
The step count for the above algorithm can be expressed as recursive formula as
given below :

tSumR(n) =

These recursive formulas are referred to as recurrence relations.

One way of solving them is to make repeated substitutions for each occurrence of
tSumR, till all occurrences disappear.
tSumR (n) = 2 + tSumR (n – 1)
= 2 + 2 + tSumR (n – 2) or 2(2) + tSumR (n – 2)
= 3(2) + tSumR (n – 3)
.
.
.
= n(2) + tSumR (0)
= 2n + 2

The step count informs the changes in the run time for a program with the changes
in the instance characteristics.
For example, in 2n + 2, if the n increases by the factor of 10, then the run time
also increases by the factor of 10. So, the runtime grows linearly in n.

The input size is one of the instance characteristics, that is frequently used.
For any problem instance, the input size is the number of words / elements
needed to describe that instance.

For example, for the problem of summing an array with n elements, the input size
is n + 1.
 Now, let us consider the problem of adding two m X n matrices a and b together.
The concerned algorithm is :
Algorithm Add(a, b, c, m, n)
{
for i:= 1 to m do
for j:= 1 to n do
c[i, j] = a[i, j] + b[i, j];
}

After introducing the count incrementing statements, the above algorithm becomes :
Algorithm Add(a, b, c, m, n)
{
for i:= 1 to m do
{ count := count + 1;
for j:= 1 to n do
{ count := count + 1;
c[i, j] = a[i, j] + b[i, j];
count := count + 1;
}
count := count + 1;
}
count := count + 1:
}
 The above algorithm can be further simplified with only count incrementing
statements as shown below :
Algorithm Add(a, b, c, m, n)
{
for i:= 1 to m do
{ count := count + 2;
for j:= 1 to n do
count := count + 2;
}
count := count + 1:
}

In this algorithm, the instance characteristics are m and n.

So, the step count is determined as

2mn + 2m + 1.

If m > n, the two for loops are interchanged to make the step count as
2mn + 2n + 1.

Now, the input size is

2mn + 2.
 Second Method :
Build a table, with the total number of steps contributed by each statement to
determine the step count of an algorithm.
For this, determine :
- The number of steps per execution (s / e) of the statement.
- The frequency (total number of times executed) of each statement.
The total contribution of each statement is obtained, by multiplying the above
two quantities.
The step count for the entire algorithm is the sum of the contributions of all the
statements.
Ex : Find the step count for the algorithm, which finds the sum of n numbers stored in
an array iteratively, using tabulation method.

Statement s/e Frequency Total Steps

Algorithm Sum(a, n) 0 - 0
{ 0 - 0
s := 0.0; 1 1 1
for j := 0 to n-1 do 1 n+1 n+1
s := s + a[j]; 1 n n
return s; 1 1 1
} 0 - 0

Total 2n + 3
Ex : Find the step count for the algorithm, which finds the sum of n numbers stored in
an array recursively, using tabulation method.

tSumR(n) =

Statement s/e Frequency Total Steps

n≤0 n>0 n≤0 n>0
Algorithm SumR(a, n) 0 - - 0 0
{ 0 - - 0 0
if (n ≤ 0) then 1 1 1 1 1
return 0.0; 1 1 0 1 0
else
return SumR(a, n-1) + a[n]; 1+x 0 1 0 1+x
} 0 - - 0 0

Total 2 2+x

Note : x = tSumR(n – 1)
Ex : Find the step count for the algorithm, which adds two m x n matrices, using
tabulation method.

Statement s/e Frequency Total Steps

Algorithm Add(a, b, c, m, n) 0 - 0
{ 0 - 0
for i:= 1 to m do 1 m+1 m+1
for j:= 1 to n do 1 m (n + 1) mn + m
c[i, j] = a[i, j] + b[i, j]; 1 mn mn
} 0 - 0

Total 2mn + 2m + 1

Once, sufficient experience is earned, the construction of the frequency table can be
avoided.

Ex : The sequence of the Fibonacci numbers is :

0, 1, 1, 2, 3, 5, 8, …
This sequence begins with f0 = 0, f1 = 1, from there onwards
fn = f n – 1 + f n – 2 , n ≥ 2
Write an algorithm to find nth Fibonacci number.
Algorithm Fib(n)
{
if (n ≤ 1) then
write (n);
else
{
fn – 2 = 0; fn – 1 = 1;
for j := 2 to n do
{
f n = f n – 1 + f n – 2;
fn - 2 = fn – 1; fn – 1 = fn;
}
write (fn);
}
}
Here, two cases exist, namely n ≤ 1 and n > 1.
In the case of n ≤ 1, the step count is 2.
n > 1, the step count is 4n + 1.

After determining the instance characteristics, which influence the step count, the
step count can be determined by using either of the above two methods.
 In some algorithms, the chosen parameters are not adequate to determine the step
count uniquely.
For these algorithms, the step counts are determined in three cases :
- Best case
- Worst case
- Average case.

The best-case step count is the minimum number of steps that can be executed
for the given parameters.
The worst-case step count is the maximum number of steps that can be
executed for the given parameters.
The average-case step count is the average number of steps executed on
instances with the given parameters.

The main purpose of determining, the step counts is to :

- Compare two algorithms that compute the same function
- Predict the growth in run time as the instance characteristics change.

In most situations, the time complexities are also expressed as

c1n2 ≤ tP(n) ≤ c2n2 or tQ(n, m) = c1n + c2m.

The break-even point is the value of n, beyond which the performances of the
algorithms switch.
Asymptotic Notation :
This notation is used to make meaningful statements about the time and space
complexities of an algorithm.
Let us assume that the functions f and g are nonnegative.

O (Big “oh”) – Notation : The function f(n) = O(g(n)) iff there exists positive
constants c and n0 such that f(n) ≤ c * g(n) for all n, n ≥ n0. (= means is)

Ex : The function 3n + 2 = O(n) as 3n + 2 ≤ 4n for all n ≥ 2.

The function 3n + 3 = O(n) as 3n + 3 ≤ 4n for all n ≥ 3.

|||ly 100n + 6 = O(n) as 100n + 6 ≤ 101n for all n ≥ 6.

10n2 + 4n + 2 = O(n2) as 10n2 + 4n + 2 ≤ 11n2 for all n ≥ 5.

1000n2 + 100n - 6 = O(n2) as 1000n2 + 100n - 6 ≤ 1001n2 for all n ≥

100.

6 * 2n + n2 = O(2n) as 6 * 2n + n2 ≤ 7 * 2n for all n ≥ 4.

3n + 2 ≠ O(1) is not less than or equal to any constant c for all n ≥ n0.
The meaning of these notations are :
O(1) is a constant.
O(n) is called linear.
O(n2) is called quadratic.
O(n3) is called cubic.
O(2n) is called exponential.
O(log n) is faster for sufficiently large n, but less than O(n).
O(n log n) is better than O(n2), but not as good as O(n).

The relation among the above are :

O(1) < O(log n) < O(n) < O(n log n) < O(n2) < O(n3) < O(2n).

According to this notation, g(n) is an upper bound on the value of f(n) for all n,
n≥n0. So, g(n) should be as small as possible for which f(n) = O(g(n)).

If f(n) = amnm + … + a1n + a0 then f(n) = O(nm).

 Ω (Omega) – Notation : The function f(n) = Ω(g(n)) iff there exist positive
constants c and n0 such that f(n) ≥ c * g(n) for all n, n ≥ n0.

Ex : The function 3n + 2 = Ω(n) as 3n + 2 ≥ 3n for all n ≥ 1.

The function 3n + 3 = Ω(n) as 3n + 3 ≥ 3n for all n ≥ 1.

|||ly 100n + 6 = Ω(n) as 100n + 6 ≥ 100n for all n ≥ 1.

10n2 + 4n + 2 = Ω(n2) as 10n2 + 4n + 2 ≥ n2 for all n ≥ 1.

6 * 2n + n2 = Ω(2n) as 6 * 2n + n2 ≥ 2n for all n ≥ 1.

3n + 3 = Ω(1)

10n2 + 4n + 2 = Ω(n)

10n2 + 4n + 2 = Ω(1)

6 * 2n + n2 = Ω(n2)

6 * 2n + n2 = Ω(n)

6 * 2n + n2 = Ω(1)
 It is observed from the above examples that, there are several functions g(n) for
which f(n) = Ω(g(n)).
Here, the function g(n) is a lower bound on the f(n). So, g(n) should be as large
a function of n as possible for which f(n) = Ω(g(n)) is true.

If f(n) = amnm + … + a1n + a0 and am > 0 then f(n) = Ω(nm).

θ (Theta) – Notation : The function f(n) = θ(g(n)) iff there exist positive
constants c1, c2, and n0 such that c1g(n) ≤ f(n) ≤ c2g(n) for all n, n ≥ n0.

Ex : The function 3n + 2 = θ(n) as 3n ≤ 3n + 2 ≤ 4n for all n ≥ 2. c1=3, c2=4, n0=2.

3n + 3 = θ(n)
10n2 + 4n + 2 = θ(n2)
6 * 2n + n2 = θ(2n)
3n + 2 ≠ θ(1)
3n + 3 ≠ θ(n2)
10n2 + 4n + 2 ≠ θ(n)
10n2 + 4n + 2 ≠ θ(1)
 The function f(n) = θ(g(n)) iff g(n) is both an upper and lower bound on f(n).

If f(n) = amnm + … + a1n + a0 and am > 0 then f(n) = θ(nm).

o (Little “oh”) – Notation : The function f(n) = o(g(n)) iff

=0

Ex : The function 3n + 2 = o(n2) since = 0.

3n + 3 = o(n log n)
6 * 2n + n2 = o(3n)
6 * 2n + n2 = o(2n log n)

ω (Little omega) – Notation : The f(n) = ω(g(n)) iff

=0
Asymptotic complexity of Sum :
Statement s/e Frequency Total Steps

Algorithm Sum(a, n) 0 - Θ(0)

{ 0 - Θ(0)
s := 0.0; 1 1 Θ(1)
for j := 0 to n-1 do 1 n+1 Θ(n)
s := s + a[j]; 1 n Θ(n)
return s; 1 1 Θ(1)
} 0 - Θ(0)

Total Θ(n)

Asymptotic complexity of SumR :

Statement s/e Frequency Total Steps

n=0 n>0 n=0 n>0
Algorithm SumR(a, n) 0 - - 0 Θ(0)
{ 0 - - 0 Θ(0)
if (n < 0) then 1 1 1 1 Θ(1)
return 0.0; 1 1 0 1 Θ(0)
else
return SumR(a, n-1) + a[n]; 1+x 0 1 0 Θ(1+x)
} 0 - - 0 Θ(0)
Total 2 Θ(1+x)
Asymptotic complexity of Add :

Statement s/e Frequency Total Steps

Algorithm Add(a, b, c, m, n) 0 - Θ(0)

{ 0 - Θ(0)
for i:= 1 to m do 1 Θ(m) Θ(m)
for j:= 1 to n do 1 Θ(mn) Θ(mn)
c[i, j] = a[i, j] + b[i, j]; 1 Θ(mn) Θ(mn)
} 0 - 0

Total Θ(mn)

The growth of the various functions, with the value of n is tabulated as below :

log n n n log n n2 n3 2n
0 1 0 1 1 2
1 2 2 4 8 4
2 4 8 16 64 16
3 8 24 64 512 256
4 16 64 256 4,096 65,536
5 32 160 1,024 32,768 4,29,49,67,296
The plot of the function values is as shown below :
DIVIDE – AND - CONQUER
 General Method
In divide-and-conquer strategy, the n inputs of a given function are
splitted into k distinct subsets, 1 < k ≤ n, resulting in k subproblems.

If the subproblems are still large, they will be further subdivided. This
process continues till the obtained subproblem is easily solvable without
further division.

These subproblems have to be solved and the subsolutions obtained are

to be combined to form the solution.

Usually, the subproblems resulting from this strategy are of the same type
as the original problem. So recursion is the suitable technique.

A control abstraction that mirrors the way an algorithm based on

divide-and-conquer will look will be written.

A control abstraction is a procedure whose flow of control is clear but the

primary operations are specified by other procedures, whose precise
meanings are left undefined
Algorithm DAndC(P)
{
if Small(P) then return S(P);
else
{
divide P into smaller instances P1, P2, ……., Pk, k ≥ 1;
Apply DAndC to each of these subproblems;
return Combine(DAndC(P1), DAndC(P2), … , DAndC(Pk));
}
}
The above algorithm is initially invoked as DAndC(P), where P is the problem to be
solved.

Small(P) is a function that determines whether the input size is small enough to
compute the answer without splitting and returns a boolean value.

Combine is a function that merges the solutions of k subproblems to get the

solution for P.

Let the size of P be n and the sizes of the k subproblems are n 1, n2, … ,nk
respectively. The computing time of DAndC is specified as recurrence relation.

(1)
Here,
- T(n) is the time taken by DAndC for an input size of n.
- g(n) is the time taken for computation on small inputs.
- f(n) is the time required for splitting P and merging the solutions of the
subproblems.

For many divide-and-conquer problems, the time complexity is specified by the

recurrences of the following form :

(2)

Here, a, b and T(1) are known and n is a power of b (i.e., n = b k).

One of the method to solve recurrence relation is the substitution method.
In this method, each occurrence of the function T on the right-hand side is
repeatedly substituted till all occurrences disappear.

Ex: Let us solve the following recurrence relation

with the values a = 2, b = 2, T(1) = 2 and f(n) = n.

T(n) = 2 T(n/2) + n
= 2 [2 T(n/4) + n/2] + n
= 4 T(n/4) + 2n
= 4 [2 T(n/8) + n/4] + 2n
= 8 T(n/8) + 3n
.
.
.
In general, T(n) = 2i T(n / 2i) + in, for any log2 n ≥ i ≥ 1.

So, in particular T(n) = 2 log n

2 T( n / 2 log n
2 ) + n log2 n for i = log2 n.

Thus, T(n) = n T(1) + n log2 n = 2n + n log2 n.

Beginning with the recurrence relation (2) and using the substitution
method, it can be shown that
T(n) = nlogb a [T(1) + u(n)]
Where
The following table gives the asymptotic value of u(n) for various values
of h(n).
h(n) u(n)
O(nr), r < 0 O(1)
Θ((log n)i), i ≥ 0 Θ((log n)i + 1 / (i + 1))
Ω(nr), r > 0 Θ(h(n))

The above table helps in obtaining the asymptotic value of T(n) for many
recurrences, resulted in analysis of divide-and-conquer algorithms.

Ex : Consider the following recurrence when n is a power of 2.

Here, a = 1, b = 2 and f(n) = c.

So, logba = 0 and h(n) = f(n) / nlogba = c = c(log n)0 = Θ((log n)0).

From the table, u(n) = Θ(log n).

Thus, T(n) = nlogba[c + Θ(log n)] = Θ(log n)

 Binary Search
Let ai, 1 ≤ i ≤ n, be a list of elements, sorted in ascending order.
Binary search is the problem of determining whether the given element x is present
in the list or not.

If x is present in the list, then determine a value j such that aj = x. If x is not in the
list, then j is set to 0.

This problem can be solved by using the divide-and-conquer technique.

Let Small(P) be true if n = 1. Now, S(P) will take the value i if x = ai; otherwise it
will take value 0. Then g(1) = Θ(1).

If P has more than one element, then it has to be divided into a new subproblems.
Pick an index q (in the range [i, l] as a middle index).
There are three possibilities:
1> x = aq: The problem P is solved.
2> x < aq: The x has to be searched in the sublist of the range [i, q – 1].
3> x > aq: The x has to be searched in the sublist of the range [q + 1, l].

The following algorithm BinSrch describes the binary search and has four inputs a[],
i, l and x.
Algorithm BinSrch(a,i,l,x)
{
if (l = i) then // If Small (P)
{
if (x = a[i]) then return i;
else
return 0;
}
else
{ // Reduce P into a smaller subproblems.
mid := [(i + l)/2)];
if (x = a[mid]) then return mid;
else if (x < a[mid]) then
return BinSrch(a, i, mid – 1, x);
else
return BinSrch(a, mid + 1, l, x);
}
}

The non-recursive algorithm for binary search is BinSearch , as given

Algorithm BinSearch(a, n, x)
{
low := 1; high := n;
while (low ≤ high) do
{
mid := [(low + high)/2];
if (x < a[mid]) then high := mid – 1;
else if (x > a[mid]) then low := mid + 1;
else return mid; }
return 0; }
Ex : Let a[1 : 14] = -15, -6, 0, 7, 9, 23, 54, 82, 101, 112, 125, 131, 142, 151.
For n = 14, the binary decision tree, which reflects the way of search,
using the algorithm BinSearch is :

The circular nodes are called internal nodes, and square nodes are called
external nodes.
The successful search terminates an internal node, and the unsuccessful
at an external node.
Theorem : If n is in the range [2k – 1, 2k), then BinSearch makes at most k
element comparisons for a successful search and either k – 1 or k
comparisons for an unsuccessful search. (The time taken for successful
search is O(log n) and for an unsuccessful search is Θ(log n)).

Proof : Let us consider the binary decision tree, which describes the
action of BinSearch on n elements.
All successful searches end at a circular node and unsuccessful searches
end at a square node.
If 2k – 1 ≤ n < 2k, then all circular nodes are at levels 1, 2, …, k and the
square nodes at k and k + 1.

The number of element comparisons needed to terminate at a circular

node at level i is i.
The number of element comparisons needed to terminate at a square
node at level i is i – 1.
The above theorem describes the worst-case time for binary search.

To determine the average behaviour, the size of the binary decision tree is
equated to the number of element comparisons in the algorithm.
The distance of a node from the root is one less than its level.
The internal path length I is the sum of the distances of all internal
nodes from the root.
The external path length E is the sum of the distances of all external
nodes from the root.
For any binary tree with n internal nodes, E and I are related by the
following formula.
E = I + 2n.
There is a simple relationship among E, I, and the average number of
comparisons in binary search.
Let As(n) be the average number of comparisons in a successful search.
Au(n) be the average number of comparisons in an unsuccessful search.

For an internal node, the number of comparisons needed is one more than
the distance of it from the root.
As(n) = 1 + I / n.

For an external node, the number of comparisons needed is equal to the

distance of it from the root.
There will be n + 1 external nodes for n internal nodes in a binary decision
tree.
Au(n) = E / (n + 1).
Using the above three formulae, As(n) can be expressed in terms of Au(n)
as shown below :
As(n) = (1 + 1/n) Au(n) – 1.

The minimum value for As(n) and Au(n) can be achieved by an algorithm
whose binary decision tree has minimum external and internal path
length.
For this, the binary decision tree should have external nodes at adjacent
levels and this is possible by a tree produced by binary search algorithm.
Since E is proportional to n log n, both As(n) and Au(n) are proportional to
log n.
The best, average and worst cases for successful and unsuccessful
searches are :

Successful Search :
Best Case : Θ(1)
Average Case : Θ(log n)
Worst Case : Θ(log n)

Unsuccessful Search :
Best, Average and Worst Cases : Θ(log n).
 Quick Sort
The sorting technique quick sort also uses divide-and-conquer strategy, but it
differs from merge sort.

In merge sort, the file a[1 : n] was divided at its midpoint into subarrays, which
were independently sorted and later merged.

But, in quick sort, the division into subarrays is made in such a way that the sorted
subarrays do not need to be merged later.

In this, the elements in a[1 : n] are rearranged such that a[i] ≤ a[j], for all i
between 1 and m and for all j between m + 1 and n for some m, 1 ≤ m ≤ n.
Now, the elements in a[1 : m] and a[m + 1 : n] are independently sorted. Merge
operation is not needed.

For rearrangement of elements, some element in a[], say t = a[s] is selected.

Now the reordering is performed such that all elements appearing before t in a[1:n]
are ≤ t and all elements appearing after t are ≥ t.

The above process of rearrangement of elements is called as partitioning.

The algorithms for partitioning and interchanging are :

Algorithm Partition(a, m, p)
{
v := a[m]; i :=m; j := p;
repeat
{
repeat
{i := i +1;
}until (a[i] ≥ v);
repeat
{j := j -1;
}until (a[j] ≤ v);
if (i < j) then Interchange(a, i, j);
}until (i ≥ j);
a[m] := a[j]; a[j] := v; return j;
}

Algorithm Interchange(a, i, j)
{
p := a[i];
a[i] := a[j]; a[j] := p;
}
The algorithm for Quick Sort is :

Algorithm QuickSort(p, q)
{
if (p < q) then // If there are more than one element
{
// divide P into two subproblems.
j := Partition(a, p, q + 1);
// j is the position of the partitioning element.
// Solve the subproblems.
QuickSort(p, j – 1);
QuickSort(j + 1, q);
// There is no need for combining solutions.
}
}

In the analysis of QuickSort algorithm, the number of element comparisons C(n) is

counted. All the other operation’s frequency count is of the same order as C(n).

The following assumptions are made :

- The n elements to be sorted are distinct.
- The partitioning element v = a[m] in the call to Partition(a, m, p) has equal
probability of being ith smallest element, 1 ≤ i ≤ p – m in a[m : p – 1].
First, obtain the worst-case value CW(n) of C(n).
The number of element comparisons in each call of Partition is at most p – m + 1.
Let r be the total number of elements in all the calls to Partition at any level of
recursion.

At the beginning, only one call is made to Partition(a, 1 , n + 1) and r = n.

At the next level, at most two calls are made Partition and r = n – 1 and so on.

So, at each level of recursion, Partition makes O(r) element comparisons. It is

observed that the value r is one less than the value of r at previous level.
Hence, CW(n) is the sum on r as r varies from 2 to n or O(n 2).

Now, let us compute the average value CA(n) of C(n).

With the earlier assumptions, the partitioning element v has an equal probability of
being ith smallest element, 1 ≤ i ≤ p – m in a[m : p – 1].
So, the two subarrays remaining to be sorted are a[m : j] and a[j + 1 : p – 1] with
probability 1 / (p – m), m ≤ j < p.

From this the obtained recurrence relation is :

(1)
The number of element comparisons required by partition on the first call is n + 1.
CA(0) = CA(1) = 0.

Multiply both sides of (1) with n to obtain

n CA(n) = n(n + 1) + 2[CA(0) + CA(1) + … + CA(n – 1)] (2)

Substitute n -1 for n in (2)

(n – 1) CA(n – 1) = n(n – 1) + 2[CA(0) + CA(1) + … + CA(n – 2)] (3)

Subtract (3) from (2) to get

n CA(n) – (n – 1) CA(n – 1) = 2n + 2CA(n – 1) (4)

Now let us solve (4)

n CA(n) = 2n + 2CA(n – 1) + (n – 1) CA(n – 1)
= 2n + (n – 1 + 2) CA(n – 1)
= 2n + (n + 1) CA(n – 1) (5)

Divide both sides of (5) by n (n + 1), results in

(6)
Repeatedly substituting in (6) for CA(n – 1), CA(n – 2), …

.
. (7)
.

Since
(7) Yields
CA(n) ≤ 2(n + 1) [loge (n + 2) – loge 2] = O(n log n).

So, the worst-case time is O(n2) and the average time is O(n log n).

Let us consider the stack space needed by the recursion.

In worst case, the maximum depth of the recursion may be n – 1.
This happens when the partition element is the smallest value is a[m : p - 1].

By using the iterative algorithm for quick sort, the stack space can be reduced to
O(log n).
In this, the smaller of the two subarrays a[p : j – 1] and a[j + 1 : q] is always
sorted first.
The second recursive call is replaced by some assignment statements and a jump to
the beginning of the algorithm.

Let S(n) be the maximum stack space needed and it can be expressed as :

So, S(n) is less than 2 log n. Thus the maximum stack space needed is O(log n).
Algorithm QuickSort2(p, q)
{
// stack is a stack of size 2 log(n);
repeat
{
while (p < q) do
{
j := Partition(a, p, q +1);
if ((j – p) < (q – j)) then
{
Add(j + 1); // Add j +1 to stack
Add(q); q := j – 1; // Add q to stack
}
else
{
Add(p); // Add p to stack.
Add(j – 1); p := j + 1; // Add j – 1 to stack
}
} // Sort the smaller subfile.
if stack is empty then return
Delete (q); Delete(p); // Delete q and p from stack.
} until (false);
}
 Merge Sort
In this sorting technique, the given list of n elements are splitted into two
sublists. (This process continues till a sublist of size 1 is produced.)
Now, each sublist is individually sorted and the resulting sorted sequences
are merged, finally to produce a single sorted sequence of n elements.

The recursive algorithm for merge sort is :

Algorithm MergeSort(low, high)
{
if (low < high) then // If there are more than one element.
{
// Divide P into subprblems.
// Find where to split the set.
mid := [(low +high)/2];
// Solve the subproblems.
MergeSort(low, mid);
MergeSort(mid + 1, high);
// Combine the solutions.
Merge(low, mid, high);
}
Algorithm Merge(low, mid, high)
{ h := low; i := low; j := mid + 1;
while ((h ≤ mid) and (j ≤ high)) do
{ if (a[h] ≤ a[j]) then
{ b[i] := a[h]; h := h +1;
} else
{ b[i] := a[j]; j := j +1;
}
i:= i + 1;
}
if (h > mid) then
for k := j to high do
{ b[i] := a[k]; i := i + 1;
}
else
for k := h to mid do
{ b[i] := a[k]; i := i + 1;
}
for k := low to high do a[k] := b[k];
}
The calls of MergeSort(1, 10) can be represented in the form the following
tree :
The calls of Merge from MergeSort algorithm can be represented in the
following tree form :

The above trees can be realized by taking an array of ten elements.

a[1 : 10] = (310, 285, 179, 652, 351, 423, 861, 254, 450, 520) and
calling MergeSort(1, 10).
Let the time taken for the merging operation is proportional to n. The
computing time for merge sort is described by the following recurrence
relation.

When n is a power of 2, i.e., n = 2k, this recurrence relation can be solved

by successive substitutions.

T(n) = 2
T(n/2) + cn
= 2
[2 T(n/4) + cn/2] + cn
= 4
T(n/4) + 2cn
= 4
[2 T(n/8) + cn/4] + 2cn
= 8
T(n/8) + 3cn
.
.
.
= 2k T(1) + kcn
= an + cn log n

It is easy to observe that if 2k < n ≤ 2k + 1, then T(n) ≤ T(2k + 1).

 T(n) = O(n log n).
 Even though the MergeSort algorithm captures divide-and-conquer technique nicely,
there are several inefficiencies:
1> This algorithm uses 2n locations. This additional n locations are needed to merge
two sorted sublists.
On the each call of Merge, the result placed into b[low : high] has to be copied back
into a[low : high].

To overcome this problem, a new field of information is to be associated with each

key.
This field is used to link the keys and any related information (records) together in
a sorted list.
Now, the merging of sorted lists is achieved by changing the link values, without
the need to move the records.

For this, an auxiliary array Link[1 : n] that contains integers in the range [0, n] is
defined along with the original array a[].
These integers are interpreted as pointers to elements of a[]. A list is a sequence of
pointers ending with zero.

Now let us consider the following Link[1 : 8] of pointers to a[1 : 8].

[1] [2] [3] [4] [5] [6] [7] [8]
Link 6 4 7 1 3 0 8 0
 The integer Q = 2 denotes the start of one list and R = 5 is the start of another list.
So, the list Q = (2, 4, 1, 6) and the list R = (5, 3, 7, 8).
These lists are interpreted as a[2] ≤ a[4] ≤ a[1] ≤ a[6] and
a[5] ≤ a[3] ≤ a[7] ≤ a[8].

2> The algorithm MergeSort uses the stack space because of recursion and this space
is proportional to log n.

This can be avoided by designing an algorithm for merge sort, that works in
bottom-up rather than top-down.
For smaller sized lists, to save stack space, it is better to use insertion sort. The
algorithm is :

Algorithm InsertionSort(a, n)
{
for j := 2 to n do
{
item := a[j]; i:= j – 1;
while (( i ≥ 1) and (item < a[i])) do
{
a[i + 1] := a[i]; i := i – 1;
}
a[i + 1] := item;
}
Now, the modified MergeSort algorithm with links and insertion sort for
the size of less than 15 size is :

Algorithm MergeSort1(low, high)

// The global array a[low : high] to be sorted.
// The auxiliary array Link[low : high] is used for sorting.
// A pointer to the beginning of the list is returned.
{
if((high – low) < 15) then
return insertionSort1(a, link, low, high);
else
{
mid := [(low + high)/2];
q := MergeSort1(low, mid);
r := MergeSort1(mid + 1, high);
return Merge1(q, r);
}
}
The modified Merge algorithm, which uses the links for sorting elements
is:
Algorithm Merge1(q, r)
{
i := 1; j := r; k := 0;
// The new list starts at link[0];
while ((i ≠ 0) and (j ≠ 0)) do
{ // While both lists are nonempty do
if (a[i] ≤ a[j]) then
{ // Find the smaller key.
link[k] := i; k := i; i := link[i];
// Add a new key to the list.
}
else
{
link[k] := j; k := j; j := link[j];
}
}
if (i = 0) then link[k] :=j;
else link[k] :=i;
return link[0];
Strassen’s Matrix Multiplication
Let A and B be two n X n matrices. So, the product matrix C = A * B is
also of size n X n.
The (i, j)th element of C is formed by multiplying the elements in the ith
row of A and jth column of B, to obtain:

Using the above formula, each element of C requires n multiplications. So,

for the construction of C, n3 multiplications are required.
 With conventional method, the total time required is (n3).

There is another way to compute the product of two n X n matrices using

the divide-and-conquer strategy.
Let us assume that the n is a power of 2, i.e., n = 2k, k is a nonnegative
integer.
If n is not a power of 2 then enough rows and columns of zeros are added
to both A & B to make the n value as a power of 2.
Let A and B be each are partitioned into four square submatrices, each
submatrix is having dimensions n/2 X n/2.
This process is continued till each submatrix is of size 2 X 2.
Now, the product AB can be computed of 2 X 2 matrices. by using the
above formula for the product

then
C11 = A11 B11 + A12 B21
C12 = A11 B12 + A12 B22
C21 = A21 B11 + A22 B21
C22 = A21 B12 + A22 B22

Since, n is a power of 2, these matrix products can be recursively

computed by the same algorithm.
To compute AB eight multiplications and four additions of n/2 X n/2 are
required. Two n/2 X n/2 can be added in time cn2 for some constant c.

So, the computing time T(n) of the resulting divide-and-conquer algorithm

is given by the recurrence relation:

After solving the above recurrence relation with the substitution

technique, T(n) = O(n3) is obtained.
Hence, there is improvement over the conventional method ((n3) versus
O(n3)).
Volker Strassen has discovered a way to compute the Cij ‘s using only 7
multiplications 18 additions or subtractions.
His method involves first computing the seven n/2 X n/2 matrices
P, Q, R, S, T, U, and V using 7 matrix multiplications and 10 matrix
additions or subtractions.
Now Cijs require an additional 8 additions or subtractions.
P = (A11 + A22) (B11 + B22)
Q = (A21 + A22) B11
R = A11 (B12 – B22)
S = A22 (B21 – B11)
T = (A11 + A12) B22
U = (A21 – A11) (B11 + B12)
V = (A12 – A22) (B21 + B22)
Now, the elements of the matrix C can be obtained as:
C11 = P + S – T + V
C12 = R + T
C21 = Q + S
C22 = P + R – Q + U
The resulting recurrence relation for T(n) is :

where a and b are constants.