DESIGN AND ANALYSIS OF ALGORITHMS

UNIT 1

NOTION OF ALGORITHM

Algorithm

An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required
output for any legitimate input in a finite amount of time. This definition can be illustrated by diagram
below.

Characteristics of an algorithm:
Input: Zero / more quantities are externally supplied.
Output: At least one quantity is produced.
Definiteness: Each instruction is clear and unambiguous.
Finiteness: If the instructions of an algorithm is traced then for all cases the algorithm must
terminates after a finite number of steps.
Efficiency: Every instruction must be very basic and runs in short time.

Time Complexity

The time needed by an algorithm expressed as a function of the size of a problem is called the time
complexity of the algorithm. The time complexity of a program is the amount of computer time it needs
to run to completion.

Space Complexity
The space complexity of a program is the amount of memory it needs to run to
completion. The space need by a program has the following components:

Example illustrating the notion of the algorithm, consider 3 methods for solving the same problem:
Computing the GCD of two integers.

These examples will help us to illustrate several important points:

 The non-ambiguity requirement for each step of an algorithm cannot be compromised.

Page 1 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

 The range of inputs for which an algorithm works has to be specified carefully.
 The same algorithm can be represented in several different ways.
 There may exist several algorithms for solving the same problem.
 Algorithms for the same problem can be based on very different ideas and can solve the problem
with dramatically different speeds.
gcd(m, n) is defined as the largest integer that divides both m and n evenly, i.e., with a remainder
of zero.

1. Euclid’s algorithm is based on applying repeatedly the equality.

gcd(m, n) = gcd(n, m mod n).
example : find gcd(60, 24)
gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12.

Algorithm:
Step 1 If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step 2 Divide m by n and assign the value of the remainder to r.
Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.

In pseudo code:

ALGORITHM Euclid(m, n)
//Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers m and n
//Output: Greatest common divisor of m and n
while n != 0 do
r ←m mod n
m←n
n←r
return m.
2. Consecutive integer checking algorithm for computing gcd(m, n)

Step 1 Assign the value of min{m, n} to t.

Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, go to Step 4.
Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer and
Stop; otherwise, proceed to Step 4.
Step 4 Decrease the value of t by 1. Go to Step 2.

3. Middle-school procedure for computing gcd(m, n)

Step 1 Find the prime factors of m.
Step 2 Find the prime factors of n.
Step 3 Identify all the common factors in the two prime expansions found in Step 1 and Step 2. (If
p is a common factor occurring pm and pn times in m and n, respectively, it should be
repeated min{pm, pn} times.)
Page 2 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Step 4 Compute the product of all the common factors and return it as the greatest common divisor
of the numbers given.

Eg: for the numbers 60 and 24, we get

60 = 2 . 2 . 3 . 5 24 = 2 . 2 . 2 . 3
gcd(60, 24) = 2 . 2 . 3 = 12.

Last procedure is much more complex and slower than Euclid’s algorithm. In addition to inferior efficiency,
the middle school procedure does not qualify, in the form presented, as a legitimate algorithm.
An algorithm for generating consecutive primes not exceeding any given integer n > 1 probably invented
in ancient Greece and is known as the sieve of Eratosthenes.

• The algorithm starts by initializing a list of prime candidates with consecutive integers from 2 to n.
• First iteration, the algorithm eliminates from the list all multiples of 2.
• Then it moves to the next item on the list, which is 3, and eliminates its multiples.
• No pass for number 4 is needed: since 4 itself and all its multiples are also multiples of 2, they were
already eliminated on a previous pass.
• The algorithm continues in this fashion until no more numbers can be eliminated from the list. The
remaining integers of the list are the primes needed.

Eg: prime numbers <=25

ALGORITHM Sieve(n)
//Implements the sieve of Eratosthenes
//Input: A positive integer n > 1
//Output: Array L of all prime numbers less than or equal to n
for p←2 to n do A[p]←p
for p←2 to √n do //see note before pseudocode
if A[p] !=0 //p hasn’t been eliminated on previous passes
j ← p * p
while j ≤ n do
A[j ]←0 //mark element as eliminated
j ←j + p
//copy the remaining elements of A to array L of the primes
i ←0
for p←2 to n do

Page 3 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

if A[p] != 0
L[i]←A[p]
i ←i + 1
return L

Fundamentals of the analysis of algorithm efficiency

Fundamentals of Algorithmic problem solving

• Understanding the problem

• Ascertain the capabilities of the computational device

• Exact /approximate soln.

• Decide on the appropriate data structure

• Algorithm design techniques

• Methods of specifying an algorithm

• Proving an algorithms correctness

• Analysing an algorithm

Page 4 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Understanding the problem

Before designing an algorithm, the most important thing is to understand the problem given. An Input to
an algorithm specifies an instance of the problem the algorithm that it solves.
Important to specify exactly the range of instances the algorithm needs to handle. Else it will work
correctly for majority of inputs but crash on some ―boundary value.
A correct algorithm is not one that works most of the time, but one that workscorrectly for all legitimate
inputs.
Ascertaining the capabilities of a computational device

After understanding need to ascertain the capabilities of the device.

The vast majority of algorithms in use today are still destined to be programmed for a computer
closely resembling the von Neumann machine—acomputer architecture. Von Neumann architectures are
sequential and the algorithms implemented on them are called sequential algorithms
Algorithms which designed to be executed on parallel computers called parallel algorithms.

Choosing between exact and approximate problem solving

Page 5 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

For exact result , use exact algorithm.For approximate result, use approximation algorithm.
An approximation algorithm can be a part of a more sophisticated algorithm that solves a problem
exactly.

Deciding on appropriate data structures

Algorithms may or may not demand ingenuity in representing their inputs.

Inputs are represented using various data structures.
Algorithm + data structures=program.

Algorithm Design Techniques and Methods of Specifying an Algorithm

An algorithm design technique (or ―strategy‖ or ―paradigm‖) is a general approach to solving problems
algorithmically that is applicable to a variety of problems from different areas of computing.
The different algorithm design techniques are: brute force approach, divide and
conquer, greedy method, decrease and conquer, dynamic programming, transform and conquer and back
tracking.
Methods of specifying an algorithm: Using natural language, in this method ambiguity problem will be
there.
The next 2 options are : pseudo code and flowchart.
Pseudo code: mix of natural and programming language. More precise than natural language
Flow chart: method of expressing an algorithm by collection of connected geometric shapes containing
descriptions of the algorithm‘s steps. This representation technique has proved to be inconvenient for large
problems.

Proving an Algorithm’s Correctness

After specifying an algorithm we have to prove its correctness.
The correctness is to prove that the algorithm yields a required result for every legitimate input in a finite
amount of time.
For example, the correctness of Euclid‘s algorithm for computing the greatest common divisor stems from
the correctness of the equality gcd(m, n) = gcd(n, m mod n), the simple observation that the second integer
gets smaller on every iteration of the algorithm, and the fact that the algorithm stops when the second
integer becomes 0.

Analyzing an algorithm

After correctness, efficiency has to be estimated.

 Time efficiency and space efficiency.

 Time: how fast the algorithm runs?

 Space: how much extra memory the algorithm needs?

Page 6 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Code the algorithm

 Writing program by using programming language.
 Selection of programming language should support the features mentioned in the design
phase.

Page 7 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Asymptotic notations and basic efficiency classes

 The efficiency analysis framework concentrates on the order of growth of an algorithm’s basic
operation count as the principal indicator of the algorithm’s efficiency.
 To compare and rank such orders of growth, computer scientists use three notations: O (big oh), Ω
(big omega), and Ө (big theta).
 t (n) and g(n) can be any nonnegative functions defined on the set of natural numbers.

Big Oh- O notation

Definition
A function t (n) is said to be in O(g(n)), denoted t (n) O(g(n)), if t (n) is bounded above by some
constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative
integer n0 such that t (n) ≤ cg(n) for all n ≥ n0.

Page 8 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Eg1 : 100n + 5 O(n2).

100n + 5 ≤ 100n + n (for all n ≥ 5) = 101n ≤ 101n2.
Thus, as values of the constants c and n0 required by the
definition Eg2 : 100n + 5 ≤ 100n + 5n (for all n ≥ 1) = 105n to
complete the proof with c = 105 and n0 = 1.

Big Omega- Ω notation

Definition A function t (n) is said to be in Ω(g(n)), denoted t (n) Ω(g(n)), if t (n) is bounded below by
some positive constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and
some nonnegative integer n0 such that t (n) ≥ cg(n) for all n ≥ n0.

Eg: n3 Ω (n2):
n3 ≥ n2 for all n ≥ 0,
i.e., we can select c = 1 and n0 = 0

Big Theta- Θ notation

Definition A function t (n) is said to be in Ө (g(n)), denoted t (n) Ө(g(n)), if t (n) is bounded both above
and below by some positive constant multiples of g(n) for all large n, i.e., if there exist some positive
constants c1 and c2 and some nonnegative integer n0 such that c2g(n) ≤ t (n) ≤ c1g(n) for all n ≥ n0.

Page 9 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Eg: To prove that ½ n(n − 1 ) Ω(n 2 ).

First, we prove the right inequality (the upper bound): ½ n (n − 1) = ½ n 2 – ½ n ≤ ½ n 2 for all n ≥ 0.
Second, prove the left inequality (the lower bound): ½ n (n − 1) = ½ n 2 – ½ n ≥ ½ n 2 – ½ n ½ n
(For all n ≥ 2) = ¼ n 2.
Hence, we can select c 2 = ¼, c 1 = ½, and n 0 = 2.

Useful property involving the asymptotic notations

THEOREM:
If t1 (n) O (g1 (n)) and t2 (n) O (g2 (n)), then
t1 (n) + t2 (n) O (max {g1 (n), g2 (n)}).
(The analogous assertions are true for the Ω and Ө notations as well.)

PROOF:
The proof extends to orders of growth the following simple fact about four arbitrary real numbers a1, b1, a2,
and b2:

If a1 ≤ b1 and a2 ≤ b2, then a1 + a2 ≤ 2 max {b1, b2}.

Since t1 (n) O (g1(n)), there exist some positive constant c1 and some nonnegative integer n1 such that
t1(n) ≤ c1 g1(n) for all n ≥ n1.
Similarly, since t2(n) O(g2(n)), t2(n) ≤ c2 g2(n) for all n ≥ n2.

Let us denote c3 = max {c1 , c2 } and consider n ≥ max{n1, n2} so that we can use
both inequalities. Adding them yields the following:
t1(n) + t2(n) ≤ c1 g1(n) + c2 g2(n) ≤ c3 g1(n) + c3 g2(n) = c3 [g1(n) + g2(n)] ≤ c3 2 max {g1(n), g2(n)}.

Hence, t1 (n) + t2 (n) O (max {g1 (n), g2 (n)}), with the constants c and n0 required by the O definition
being 2c3 = 2 max {c1, c2} and max {n1, n2}, respectively.

It implies that the algorithm’s overall efficiency is determined by the part with a higher order of growth,
i.e., its least efficient part:

Eg: A sorting algorithm used in the first part makes no more than ½ n(n − 1) comparisons (and hence is in
O(n2)) while the second part makes no more than n − 1 comparisons (and hence is in O(n)), the efficiency
of the entire algorithm will be in O(max{n2 , n}) = O(n2).

Page 10 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Using limits for comparing orders of growth

Though the formal definitions of O, Ω and Ө are indispensable for proving their abstract properties, they
are rarely used for comparing the orders of growth of two specific functions.
A much more convenient method for doing so is based on computing the limit of the ratio of two functions.

The limit-based approach is often more convenient than the one based on the definitions because it can take
advantage of the powerful calculus techniques developed for computing limits, such as L’Hospital’s rule

And Stirling’s formula

Eg1 : ½ n(n-1) and n

½ n(n − 1) Ө(n2).

Eg2 : log2 n and √n.

Eg3 : n! and 2n

2n grows very fast, n! Grows still faster. n! Ω(2n).

Page 11 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

MATHEMATICAL ANALYSIS OF NONRECURSIVE ALGORITHMS

Analyzing the time efficiency of non-recursive algorithms.

General Plan for Analyzing the Time Efficiency of Non-recursive Algorithms

 Decide on a parameter(s) indicating an input’s size.
 Identify the algorithm’s basic operation.
 Check whether the number of times the basic operation is executed depends only on the size of an
input. If it also depends on some additional property, the worst-case, average-case, and, if necessary,
best-case efficiencies have to be investigated separately.
 Set up a sum expressing the number of times the algorithm’s basic operation is executed.
 Using standard formulas and rules of sum manipulation, either find a closed form formula for the
count or, at the very least, establish its order of growth

Two basic rules of sum manipulation:

Page 12 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Two summation formulas:

.
Eg1: Finding the value of the largest element in a list of n numbers.

ALGORITHM MaxElement(A[0..n − 1])

//Determines the value of the largest element in a given array
//Input: An array A [0...n − 1] of real numbers
//Output: The value of the largest element in A
maxval ←A[0]
for i ←1 to n − 1 do
if A[i]>maxval
maxval←A[i]
return maxval.

 The measure of an input’s size here is the number of elements in the array, i.e., n.
 The operations that are going to be executed most often are in the algorithm’s for loop called the
basic operation.
 There are two operations in the loop’s body: the comparison A[i] > maxval and the assignment
maxval← A[ i]
 The comparison operation is the basic operation
 The algorithm makes one comparison on each execution of the loop, which is repeated for each
value of the loop’s variable i within the bounds 1 and n − 1,inclusive. Therefore, we get the
following sum for C(n):

EG2: To check whether all the elements in a given array are distinct.

ALGORITHM UniqueElements(A[0..n − 1])

//Determines whether all the elements in a given array are distinct
Page 13 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

//Input: An array A[0..n − 1]

//Output: Returns ―true‖ if all the elements in A are distinct
// and ―false‖ otherwise
for i ←0 to n − 2 do
for j ←i + 1 to
n − 1 do if A[i]=
A[j ]
return false
return true.

 The natural measure of the input’s size here is n.

 Since the innermost loop contains a single operation (the comparison of two elements), it is the
algorithm’s basic operation.

EG3:Matrix multiplication

ALGORITHM MatrixMultiplication(A[0 ..n − 1 , 0 ..n − 1] , B[0 ..n − 1 , 0 ..n − 1] )

//Multiplies two square matrices of order n by the definition-based algorithm

//Input: Two n × n matrices A and B
//Output: Matrix C = AB for
i ←0 to n − 1 do for j
←0 to n − 1 do
C [ i, j ]←[0 . 0 ]
for k←0 to n − 1 do
C[ i, j ]← C[ i, j ]+ A[ i, k] * B[ k, j]
return C.

 There is just one multiplication executed on each repetition of the algorithm’s innermost loop, which is
governed by the variable k ranging from the lower bound 0 to the upper bound n − 1 . Therefore, the
number of multiplications made for every pair of specific values of variables i and j is

Total number of multiplications M(n) is expressed by the following triple sum:

Page 14 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Estimating the running time of the algorithm on a particular machine, by the product T
(n) ≈ cm M(n) = cm n 3 . where cm is the time of one multiplication on the machine
.
spent on the additions, too: T (n) ≈ cm M(n) + ca A(n) = cm n 3 + ca n 3 = (cm + ca)n 3.
ca is the time for one addition.

EG4:To find the number of binary digits in the binary representation of a positive decimal integer.

ALGORITHM Binary(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary
representation
count ←1
while n > 1 do
count ←count + 1
n←_n/2_
return count.

 The number of times the comparison n> 1 will be executed is actually log2 n + 1 : the number of bits in
the binary representation of n.

MATHEMATICAL ANALYSIS OF RECURSIVE ALGORITHMS

General Plan for Analyzing the Time Efficiency of Recursive Algorithms

1. Decide on a parameter (or parameters) indicating an input’s size.

2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation is executed can vary on different inputs of the
same size; if it can, the worst-case, average-case, and best-case efficiencies must be investigated
separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number of times the basic
operation is executed.
5. Solve the recurrence or, at least, ascertain the order of growth of its solution.

EG1: Compute the factorial function f(n) = n! For an arbitrary nonnegative integer n.

ALGORITHM F(n)
//Computes n! recursively

Page 15 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

//Input: A nonnegative integer n

//Output: The value of n!
if n = 0 return 1
else return F(n − 1 ) * n.

 The basic operation of the algorithm is multiplication,5 whose number of executions we denote M(n).
Since the function F(n) is computed according to the formula F(n) = F(n − 1 ) . n for n > 0 , 
The number of multiplications M(n) needed to compute it must satisfy the equality

 Indeed, M(n − 1 ) multiplications are spent to compute F(n − 1 ), and one more multiplication is needed
to multiply the result by n.
 The last equation defines the sequence M(n) that we need to find. This equation defines M(n) not
explicitly, i.e., as a function of n, but implicitly as a function of its value at another point, namely n −
1.
Such equations are called recurrence relations or, for brevity, recurrences.
initial condition that tells us the value with which the sequence starts.
if n==0 return 1.
This tells us two things:
 since the calls stop when n = 0 , the smallest value of n for which this algorithm is executed and
hence M(n) defined is 0.
 by inspecting the pseudocode’s exiting line, we can see that when n = 0 , the algorithm performs
no multiplications.
 M(n) = M(n − 1 ) + 1 for n > 0 ,
M( 0 ) = 0 .
 The first is the factorial function F(n) itself; it is defined by the recurrence F(n) = F(n − 1 ) . n for
every n > 0 F( 0 ) = 1 .
 From the several techniques available for solving recurrence relations, we use what can be called the
method of backward substitutions. M(n) = M(n − 1 ) + 1
substitute M(n − 1 ) = M(n − 2 ) + 1
= [ M(n − 2 ) + 1]+ 1
= M(n − 2 ) + 2
substitute M(n − 2 ) = M(n − 3 ) + 1
= [ M(n − 3 ) + 1]+ 2
= M(n − 3 ) + 3 .
 General formula for the pattern: M(n) = M(n − i) + i.
 substitute i = n, then :
M(n) = M(n − 1 ) + 1= . . . = M(n − i) + i = . . . = M(n − n) + n = n.

EG2: Tower of hanoi puzzle

Page 16 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

 n disks of different sizes that can slide onto any of three pegs. Initially, all the disks are on the first peg
in order of size, the largest on the bottom and the smallest on top. The goal is to move all the disks to
the third peg, using the second one as an auxiliary, if necessary. We can move only one disk at a time,
and it is forbidden to place a larger disk on top of a smaller one.
We first move recursively n − 1 disks from peg 1 to peg 2 (with peg 3 as auxiliary), then move the
largest disk directly from peg 1 to peg 3, and, finally, move recursively n – 1 disks from peg 2 to peg 3
(using peg 1 as auxiliary). Of course, if n = 1 , we simply move the single disk directly from the source
peg to the destination peg.
 The number of moves M(n) depends on n only, and we get the following recurrence equation for it:
M(n) = M(n − 1 ) + 1+ M(n − 1 ) for n > 1 .
With the initial condition M( 1 ) = 1.

Page 17 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

 Total number of calls made by the tower of Hanoi problem

EG3: Number of binary digits in a binary representation

ALGORITHM BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec(n/ 2 ) + 1.

 The number of additions made in computing BinRec(n/ 2 ) is A(n/ 2 ), plus one more addition is
made by the algorithm to increase the returned value by 1. This leads to
the recurrence
A(n) = A(n/ 2 ) + 1 for n > 1 .
 The initial condition is A( 1 ) = 0 .
Due to the presence of n/2, the standard approach to solving such a recurrence is to solve it only
for n = 2 k.

Page 18 of 19
 DESIGN AND ANALYSIS OF ALGORITHMS 2022

Page 19 of 19