0% found this document useful (0 votes)
35 views66 pages

Lecture 1-4 Intro To Algo, Running Time, Asymptotic and Recurrence

Uploaded by

Vernika Sharma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
35 views66 pages

Lecture 1-4 Intro To Algo, Running Time, Asymptotic and Recurrence

Uploaded by

Vernika Sharma
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 66

Lecture 1: Introduction

Design & Analysis of Algorithms

Rajesh Kumar Tripathi


Associate Professor, GLA University, Mathura
Introduction
 Algorithm
 Algorithm characteristics
 General steps to develop an algorithm
 Analyzing parameters
Algorithm

 An algorithm is a sequence of computational steps that


transform input to desired output.
(Thomas H. Coreman)

 An algorithm is a finite set of instructions that, if followed,


accomplishes a particular task
(Sartaj Sahni)
Criteria of algorithm
 All algorithms must satisfy the following five criteria:
 Input
 Output
 Definitiveness
 Finiteness
 Effectiveness
Algorithm…
 Algorithm_name <(parameter list)>
1. Algorithm_Max(A, n) //A is an array of size n
{
2. Result=A[1];
3. for i=2 to n do
4. if(A[i] > Result) then Result=A[i];
5. return Result;
}
General steps to develop an Algorithm
 There are four steps:
 To devise an algorithm
 Validate an algorithm
 Analyze an algorithm
 Test an algorithm
 Debugging
 Profiling
Lecture 2: Analyzing Parameters

Design & Analysis of Algorithms

Rajesh Kumar Tripathi


Assistant Professor, GLA University, Mathura
Analyzing parameters
 Space Complexity
 Time Complexity
Space Complexity
 This is an amount of memory requirement for an algorithm to
completion of its execution.
Space Complexity…
 Space Complexity:
 Fixed part:
 It is independent of characteristics of inputs & outputs.
 It includes:
 instruction space,
 space for simple variables and
 fixed size component variables.

Algorithm abc(a, b, c)
{
Return (a+b+b*c+(a+b-c)/(a+b)+4.0);
}

Space is computed for a+b+b*c+(a+b-c)/(a+b)+4.0


Space Complexity…
 Space Complexity:
 Variable part:
 A variable part that consists of space needed by
component variables whose size is dependent on the
particular problem instance.
// n is the length of array a[ ]
sum(a[], n)
{
x=0
for i=1 to n
x = x + a[i]
return(x);
}
Time Complexity
 Time complexity of an algorithm is the amount of computer
time it needs to run the completion.
Compute complexity
A()
{ int i;
for i=1 to n
pf(“Hello”);
}
Complexity: O(n)
A()
{int i, j;
for i=1 to n
for j=1 to n
pf(“Hello”);
}
Complexity: O(n2)
Running Time of Algorithm
1. Get a positive integer from input Time Occurrence

2. if n>10 t1 1

3. print “This might take a while…” t2 1

4. for i=1 to n do t3 1

5. for j=1 to i do t4 n+1

6. print i*j; t5 2t5+3t5+4t5+…+(n+1)t5

7. Print “done”; t6 t6+2t6+3t6+……+nt6


t7 1
Running Time of Algorithm
In this program, line 1,2,3,7 will take t1,t2,t3,t7 time with one
occurrence.
Running Time of Algorithm
Statement 4 will take t4 time with (n+1) occurrence.

Statement 5 will take:


=2t5+3t5+4t5+…+nt5+(n+1)t5+t5-t5
=t5[(1/2)(n2+3n+2)]-t5
=(1/2)(n2+3n)t5
Running Time of Algorithm
Statement 6 will take
=t6+2t6+3t6+……+nt6=t6[(1/2)(n2+n)]
Total time
=t1+t2+t3+t7+(n+1).t4+(1/2).(n2+3n).t5+t6.(1/2).(n2+n)
=t1+t2+t3+t7+t4+(t4+3/2.t5+t6).n +……
(1/2.n2+1/2.n2)(t5+t6)
=t1+t2+t3+t4+t7+n.[t4+3/2.t5+t6]+n2.[t5+t6]

It is in the form of quadratic equation.


Conclusion
 Learned to compute the running time of an algorithm
 Analyses of the algorithm can be performed by computing
running time of algorithm
Asymptotic Notation
 Notation is used to describe the asymptotic running time of
an algorithm are defined in terms of functions whose domains
are set of natural numbers N={0,1,2,3,….}.
 Such notations are convenient for describing the worst case
running time function T(n) which is usually defined only on
integer input size.
Asymptotic Notation…
 Practical Significance
 Big Omega :
 Best case
 Never achieve better than this
 Big Theta:
 Average case
 Big O:
 Worst case
 Upper bound
 Time must not exceed
Asymptotic Notation…
10 12 35 11 40 52 22 21 25 50

0……………………………………………………………….n-1
If element=10 , then best case Ω(1)

If element=40, then average case Θ(n/2)

If element=50, then worst case O(n)


Asymptotic Notation…
 The Big-Oh (O) Notation
 Asymptotic upper bound

 f(n)=O(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) ≤ c g(n) for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.


 Used for worst case analysis
Input size
Asymptotic Notation…
 The Big-Oh (O) Notation
 Asymptotic upper bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n)= c < ∞
Then
f(n) € O(g(n))

Input size
Asymptotic Notation…
 For example: f(n) = 3n+2 g(n)=n
 f(n)= O(g(n))
f(n) ≤ c. g(n)
3n+2 ≤ c . n
c=4 , n0 = 2
Asymptotic Notation…
 For example: f(n) = 3n+2 g(n)=n
 f(n)= O(g(n))
limn->∞f(n)/g(n)= c < ∞
= limn->∞(3n+2)/n
= limn->∞(3+2/n)
= limn->∞(3+2/ꝏ)
= 3+ 2/(1/0)
= 3+ 2*0/1
= 3+0
=3<ꝏ
Hence, we can say f(n)=O(g(n))
f(n)=O(n)
Asymptotic Notation…
 Learned about the asymptotic notation
 Categories worst, average and best cases
 Learned worst case complexity representation using Big Oh
notation
Asymptotic Notation…
 The Big-Omega (Ω) Notation
 Asymptotic lower bound

 f(n)= Ω(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) ≥ c g(n) ≥ 0 for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.


 Used for best case running time
or lower bound of algorithmic problem. Input size
Asymptotic Notation
 The Big-Omega (Ω) Notation
 Asymptotic lower bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n) > 0
Then
f(n) € Ω(g(n))

Input size
Asymptotic Notation
 For example: f(n) = 3n+2 g(n)=n
 f(n)= Ω(g(n))
f(n) ≥ c. g(n)
3n+2 ≥ c . n
c=1, n0 = 1 n ≥ 1
Asymptotic Notation
 The Big-Theta (Θ) Notation
 Asymptotically tight bound

 f(n)= Θ(g(n)) , if there exists

Running time
constants c1 & c2 and n0 , s.t.
c1. g(n) ≤ f(n) ≤ c2. g(n)
for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.


Input size
 Used for average case running time
Asymptotic Notation
 The Big-Theta (Θ) Notation
 Asymptotically tight bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n) < ∞
Then,
f(n) € Θ(g(n))

Input size
Asymptotic Notation
 For example: f(n) = 3n+2 g(n)=n
 f(n) = Θ (g(n))
 c1.g(n) ≤ f(n) ≤ c2. g(n)
f(n) ≤ c2 . g(n)
3n+2 ≤ c2 . n
c2=4 , n0 ≥ 1

f(n) ≥ c1. g(n)


3n+2 ≥ c1. n
c1=1, n0 ≥ 1
Asymptotic Notation
 The Little-oh (o) Notation
 Asymptotic upper bound

 f(n)=o(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) < c g(n) for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.

Input size
Asymptotic Notation
 The Little-oh (o) Notation
 Asymptotic upper bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n)= 0
Then
f(n) € o(g(n))

Input size
Asymptotic Notation
 The Little-omega (Ω) Notation
 Asymptotic lower bound

 f(n)= Ω(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) > c g(n) ≥ 0 for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.

Input size
Asymptotic Notation
 The Little-omega (Ω) Notation
 Asymptotic lower bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n) = ∞
Then
f(n) € Ω(g(n))

Input size
Introduction
 Recurrence
 Solving Methods
 Substitution Method
 Master Method
 Recursion Tree
 Iterative Method
Introduction
 Recurrence:
 A recurrence is an equation or inequality that describes a
function in terms of its value on smaller inputs.
 An algorithm contains a recursive call to itself, we can often
describe its running time by a recurrence equation or
recurrence.
 Different forms:
 T(n)= 2 T(n/2)+f(n)
 T(n)=T(n-1)+1
 T(n)=T(2n/3)+T(n/3)+f(n)
Recurrence
 Substitution Method
 Guess a bound and then use mathematical induction to
prove our guess correct.
Recurrence
 Substitution Method
 The substitution method for solving recurrences comprises
two steps:
 Guess the form of the solution.
 Use mathematical induction to find the constants and show
that the solution works.
Recurrence
 Substitution Method
T(n)=2T(n/2)+n
Guess the solution is: O(n log n)
Prove that T(n)<= c. (n log n) for c>0
T(n)=2T(n/2)+n………………………………..(1)
Compute for T(n/2)<= c. (n/2).log(n/2) and put into (1)
T(n)<= 2. c. (n/2).log(n/2) +n
T(n)<= c. n. log(n/2) +n
T(n)<= c.n. log n –c.n.log2 +n
T(n)<= c.n.log n –c.n + n for c>=1
Recurrence
 Substitution Method
T(n)<= c.n.log n –c.n + n for c=1
T(n)<= 1.n.log n -1.n +n= n.log n
T(n)<= 2. n. log n -2. n +n = n.logn + n.logn –n
.
.
.
T(n)=O(n log n)
Recurrence
 Substitution Method
T(n)=T(n/3)+T(2n/3)+d. n
Guess the solution is: O(n log n)
Prove that T(n)<= c. (n log n) for c>0
T(n)=T(n/3)+T(2n/3)+n………………………………..(1)
Compute for T(n/3)<= c. (n/3).log(n/3) and
T(2n/3)<=c. (2n/3). log(2n/3) put into (1)
T(n)<= c. n/3. log(n/3)+ c. 2n/3. log(2n/3) + d. n
T(n)<= c. n/3. log n – c. n/3. log 3 + c. 2n/3.log 2n – c. 2n/3
log 3 + d. n
T(n)<= c. n/3. log n – c. n/3. log 3 + c. 2n/3.log 2 + c. 2n/3 log n
– c. 2n/3 log 3 + d. n
Recurrence
 Substitution Method
T(n)<= c. n/3. log n – c. n/3. log 3 + c. 2n/3.log 2 + c. 2n/3 logn
– c. 2n/3 log 3 +d. n
<= c.n.log n –c.n log 3 + c. 2n/3 + d. n
<= c. n. log n – c.n.(log 3 -2/3) + d.n
for c>= d/(log3 – 2/3)

Thank You
Keep Learning…
Design & Analysis of Algorithms

Recurrence-Master Method

Rajesh Kumar Tripathi


Associate Professor

Dept. of Computer Engineering & Applications, IET,


GLA University, Mathura
Introduction
 Recurrence
 Solving Methods
 Substitution Method
 Master Method
 Recursion Tree
 Iterative Method
Recurrence

Recurrence
 Master Method
 1. T(n) = 9 T(n/3) + n
Compare with standard formula T(n) = a T(n/b) + f(n)
a=9, b=3 and f(n)=n
Compute n(logb(a)) = n(log3 9) = n2 = n2-E

It is the form of I Case then T(n)=ѳ(n2)


Recurrence
 Master Method
 1. T(n) = T(2n/3) + 1
Compare with standard formula T(n) = a T(n/b) + f(n)
a=1, b=3/2 and f(n)=n0
Compute n(logb(a)) = n(log3/2 1) = n0

It is the form of II Case then T(n)=ѳ(lgn)


Recurrence
 Master Method
 1. T(n) = 3T(n/4) + n log n
Compare with standard formula T(n) = a T(n/b) + f(n)
a=3, b=4 and f(n)=n log n
Compute n(logb(a)) = n(log4 3) = n0.793+E
Check a f(n/b)<=c.f(n)

3. n/4 . log n/4 <= c. n log n


(3/4) n. log n/4 <= c. n.logn for c=3/4

It is the form of III Case then T(n)=ѳ(f(n))= ѳ(nlogn)


Recurrence

Thank You
Keep Learning…
Design & Analysis of Algorithms

Recurrence-Recursion Tree

Presented by:
Rajesh Kumar Tripathi
Associate Professor
Dept. of Computer Engineering & Applications, IET,
GLA University, Mathura
Introduction
 Recurrence…
 Solving Methods…
 Substitution
 Master Method
 Recursion Tree
 Iterative Method
Recurrence

Recurrence
 Recursion Tree…
 T(n)= 3 T(n/4) + c n2
Recurrence
 Recursion Tree…
 T(n)= 3 T(n/4) + c n2

Total=O(n2)
Recurrence

1
 n log 4
2 3

3 cn
1
16

Thank You
Keep Learning…
Design & Analysis of Algorithms
(BCSC0012)

Iterative Method

Presented by:
Rajesh Kumar Tripathi
AssociateProfessor
Dept. of Computer Engineering & Applications, IET,
GLA University, Mathura
Introduction
 Recurrence…
 Solving Methods…
 Substitution Method
 Master Method
 Recursion Tree
 Iterative Method
Recurrence
 Iterative Method…T(n)= 2 T(n/2)+ n s.t T(n/2)=2. T(n/2/2)+n/2
T(n)= 2 T(n/2)+ n
T(n)= 2 T(n/2)+ n ………………………………………………….(1)
T(n/2)= 2 T(n/2/2)+ n/2 = 2. T(n/22)+n/2
T(n)= 2 (2. T(n/22)+n/2 )+ n
T(n)= 22 . T(n/22) +n + n
Recurrence
 Iterative Method…T(n)= 2 T(n/2)+ n T(n/22)=2. T(n/22/2)+n/22
T(n)= 22 . T(n/22) +n + n ………………………………………..(2)
T(n/22) = 2 .T(n/22/2)+n/22
T(n/22)=2. T(n/23)+n/22 ……………………………………....(3)
T(n)= 22. (2.T(n/23)+n/22) + n + n
T(n)= 23. T(n/23) + 22. n/22 +n + n
T(n)= 23. T(n/23) + n +n + n
T(n)= 23. T(n/23) +3.n
.
.
T(n)= 2i. T(n/2i) +…+ i.n …………………………………….........(4)
Recurrence

Recurrence

Complexity of Algorithm
A()
{
int i=1,s=1;
while(s<=n)
{ i++;
s=s+i;
pf(“Hello”);
}
}
s: 1 3 6 10 15 21 ……….n
i: 1 2 3 4 5 6 ………..k
k*(k+1)/2>n
k=O(n1/2 )

You might also like