Algorithm Design and

Analysis
Lecture 2: Algorithm Analysis
 Types of Analysis
• If we have an algorithm for a problem and want to know on what inputs the algorithm is taking less time
(performing well) and on what inputs the algorithm is taking huge time.
• Since an algorithm can be represented in the form of an expression, this means we represent the
algorithm with multiple expressions:
• one for case where it is taking the less time (the best case)
• other for case where it is taking the more time (the worst case)
To analyze an algorithm we need some kind of syntax and that forms the base for asymptotic
analysis/notation. There are three types of analysis:
• Worst case
• Defines the input for which the algorithm takes huge time.
• Input is the one for which the algorithm runs the slower.
• Best case
• Defines the input for which the algorithm takes lowest time.
• Input is the one for which the algorithm runs the fastest.
• Average case
• Provides a prediction about the running time of the algorithm
• Assumes that the input is random
 Asymptotic Notation?

• Having the expressions for best case, average case and worst case.
• we need to identify the upper bound, lower bounds.
• In order to represent these upper bound and lower bounds we need some syntax.
• Let us assume that the given algorithm is represented in the form of function f(n)
 Big-O Notation
• This notation gives the tight upper bound of the
given function. It is represented as f(n) = O(g(n))
• That means, at larger values of n, the upper bound
of f(n) is g(n).
• For example, if n4+100n2+10n+50 is the given
algorithm, then n4 is g(n).
• That means, g(n) gives the maximum rate of growth for
f(n) at larger values of n.
• discard lower values of n.
• O(g(n)) = {f(n): there exists positive constants c,
n0 such that 0≤f(n) ≤cg(n) for all n>n0} Note:
Analyze the algorithms at larger values of
• In the opposite figure, n0 is the point from which only n.
we need to consider the rate of growths for a given What this means is, below n0 we do not care
for rate of growth.
algorithm.
• Below n0 the rate of growth could be different.
 Omega Ω Notation
• Similar to O discussion, this notation gives
the tighter lower bound of the given
algorithm.
• It is represented as f(n) = Ω (g(n)).
• That means, at larger values of n, the
tighter lower bound of f(n) is g(n).
• For example, if n2+10n+50 , g(n) is Ω(n2).
• Ω(g(n)) = {f(n): there exists positive
constants c, n0 such that 0≤cg(n) ≤f(n)
for all n>n0}
 Theta ΘNotation
• Θ( f ), big theta, represents the class of
functions that grow at the same rate as the
function f.
• This means that for all values of n greater than
some threshold n0, all of the functions in Θ( f )
have values that are about the same as f.
• Θ(g(n)) = {f(n): there exists positive
constants c1, c2, n0 such that 0≤c2g(n) ≤f(n)
≤c1g(n) for all n>n0}
• When we consider algorithms, we will be
interested in finding algorithms that might do
better than the one we are considering.
• So, finding one that is in big theta (in other words, is
of the same complexity) is not very interesting.
• Because we are interested in efficiency, Ω will not be of much interest to us
because Ω(n2 ), for example, includes all functions that grow faster than n2
including n3 and 2n.
• When we consider algorithms, we will be interested in finding algorithms
that might do better than the one we are considering. So, finding one that is
in big theta (in other words, is of the same complexity) is not very
interesting.
• Big-oh is the class that will be of the greatest interest. Considering two
algorithms, we will want to know if the function categorizing the behavior of
the first is in big oh of the second. If so, we know that the second algorithm
does no better than the first in solving the problem.
• Since Big Oh describes how complex the algorithm becomes as the size of the data (n)
increases
• it is mainly affected by loops

Linear Loops →n
for ( int i = 0; i < n; i ++ ) …… (or i-- )
--------------
Nested Loops → n2
for ( int i = 0; i < n; i ++ )
for ( int j = 0; j < n; j ++ )
………….
Logarithmic Loops → log n
for ( int i = 1; i < n; i = i*2 ) ……. ( or i = i/2)
 Divide and Conquer
• Divide and conquer algorithms can provide
a small and powerful means to solve a
problem. DivideAndConquer( data, N, solution )
• When we count comparisons that occur in if (N ≤ SizeLimit) then
loops, we only need to determine how DirectSolution( data, N, solution )
many comparisons there are inside the loop else
and how many times the loop is executed. DivideInput( data, N, smallerSets,
• This is made more complex when a value of smallerSizes, numberSmaller )
the outer loop influences the number of for i = 1 to numberSmaller do
passes of an inner loop. DivideAndConquer(smallerSets[i],
• When we look at divide and conquer smallerSizes[i], smallSolution[i])
algorithms, it is not clear how many times a end for
task will be done because it depends on the CombineSolutions(smallSolution, numberSmaller,
recursive calls and perhaps on some solution)
preparatory and concluding work. end if
• It is usually not obvious how many times
the function will be called recursively.
 Divide and Conquer
recurrence relations
• Recurrence relations can be directly
derived from a recursive algorithm
• Recurrence relations are in a form
that does not allow us to quickly T(n) = 2T(n – 2) – 15
determine how efficient the T(2) = 40
algorithm is. T(1) = 40
• To do that we need to convert the T(n) = 4T(n ⁄ 2) – 1
set of recursive equations into what T(4) = 4
is called closed form by removing T(3) = 4
the recursive nature of the T(2) = 4
T(1) = 4
equations.
• This is done by a series of repeated
substitutions until we can see the
pattern that develops.
 Divide and Conquer
time analysis of a recurrence algorithm
• Recursive Fibonacci Sequence – time
complexity analysis
Assume that each operation takes one unit of time
• Fibonacci sequence given by 0 1 1 2 3 5 8 . . . T(n) = T(n-1) +T(n-2) + c c=4
. T(0) = T(1) = 1
• Any element is evaluated by the T(n-1) ≈ T(n-2) T(n-1) >T(n-2)
summation of its two previous elements lower bound
T(n) = 2T(n-2) +c
Fb(n) =2{ 2T(n-4) +c} +c
{ 1 = 4T(n-4) +3c
if n<= 1 = 8T(n-6)+ 7c
= 16T(n-8)+15c
return 1 = 2kT(n-2k) + (2k-1)c
else n-2k =0
return fb(n-1)+ fb(n-2) k = n/2
T(n) = 2n/2T(0) + (2n/2-1)c Assignment :
} 1 1 higher bound
= (1+c) 2n/2 - c