Analysis of Algorithm

Algorithm:
It is the sequence of steps to solve a problem that will take input and will
produce at least one output.
Represented:
 Pseudo code (Generic, abstract)
 Flowchart (Symbolic, execution flow)
 Program (Concrete rep language, machine understanding)

1. Accuracy: (Desired)
In terms of testing | Blackbox, Whitebox

Given input Know output Software output

0 1 0
5 120 120

Test Case  Requirements

Unit testing: check one module
Integration:
f (int, int)
f (float, float)

2. Efficiency:
 Time
 Space
 Empirical Analysis
 Analytical Analysis

Add 1st 100 natural number

Algo 1: Algo 2:  Constant Complexity
Sum = 0  S1 Display 100*(100+1)/2
S2 S3 S4 S1 S2 S3 S4
for (i=1; i<100; i++) = 4 B. Op (Basic Operations)
sum += i;  S5
display sum;  S6

S1 + S2 + S3 + S4 + S5 + S6
1 + 1 + 10 + 10 + 100 + 1
0 0
1 + 1 + 1 + 2 + 2 + 1
1 + 1 + 10 + 20 + 200 + 1
0 0

Algo 1 Algo 2
0.45 0.45
0.4 0.4
0.35 0.35
0.3 0.3
0.25 0.25
Time

Time

0.2 0.2
0.15 0.15
0.1 0.1
0.05 0.05
0 0
40 60 80 100 120 140 160 180 200 220 40 60 80 100 120 140 160 180 200 220
Basic Operations Basic Operations
Empirical Analysis:
 It implemenets as Time and cost
 Differences:
o In terms of Hardware (Processor, RAM, I/O)
o In terms of Software (OS tools (Visual Studio, Turbo))
o In terms of Language (Java, C++,Assembly) plus Programming Style
(OOP, Structure).
o In terms of network (Bandwidth).
o In terms of system state.

Analytical Analysis:
It calculates unit of time in terms of Basic operations.

Sequential:

sum = 0; 1
sum = sum +10; 2
sum = sum * 2; 2
display sum; 1
= 6 B. Op

Conditional:
If-else is more powerful than switch statement
Range is given in if-else.
Example:

a = 2;  1 a = 20;  1

if (a > 10)  ½ B. Op if (a > 10)  1

{ {
 1
sum = 0; sum = 0;
sum = sum + 10; sum = sum + 10;  2
sum = sum * 2; sum = sum * 2;  2

} display sum;  1

display sum; }
 1
sum = 10;
display sum;  1

Best Case: 1+1+2

Worst Case: 1 + 1 + 6

Loop:
for ¿
cout << “Ali”;

Method 1:
¿ 1+n+2 n+ n
¿ 1+4 n
¿ 4 n=O(n)

Method 2:
n
¿ 1+ ∑ (1+ 2+ 1)
i=1

n
Example:
for ¿
for ¿
for ¿
cout << “Ali”;

Solution:

n
n(n+1)(2n+ 1)
∑ i 2= 6
i=1

n n n
¿ 1+ ∑ (1+ 2+ 1+ ∑ (1+2+1+ ∑ (1+2+1)))
i=1 j =i k= j

n n n
¿ 1+ ∑ (4+ ∑ (4+ ∑ (4)))
i=1 j =i k= j

n n
¿ 1+ ∑ (4+ ∑ (4+ 4(n− j+ 1)))
i=1 j =i

up
∴ ∑ c=up−low +1
low

n n
¿ 1+ ∑ (4+ ∑ (4+ 4 n−4 j+4)¿¿)
i=1 j =i

n n
¿ 1+ ∑ (4+ ∑ (8+4 n−4 j)¿ ¿)
i=1 j =i

n n n n
¿ 1+ ∑ (4+ ∑ 8+ ∑ 4 n−∑ 4 j¿) ¿ ¿
i=1 j =i j=i j =i

n i

n
∴ ∑ j−∑ j
n ( n+1 )−i(i+1)
¿ 1+ ∑ (4+ 8 n−8i+8+ 4 n −4∋+ 4 n−4 (
2 j=1 j =1
))
i=1 n 2n
¿ 1+ ∑ (4+ 8(n−i+1)+ 4 n(n−i+1)−4 ∑ j)
i=1 j=i

n n
¿ 1+ ∑ (4+ 8 n−8i+8+ 4 n 2−4∋+ 4 n)−4 ∑ j¿ ¿
i=1 j=i
 n
¿ 1+ ∑ (4+ 8 n−8i+8+ 4 n 2−4∋+ 4 n−2 n2−2 n+2 i 2+2 i)
i=1

n
¿ 1+ ∑ (12+ 10 n−6 i+2 n2−4∋+2 i2 )
i=1

n n n n n n
¿ 1+ ∑ 12+ ∑ 10 n−∑ 6 i+ ∑ 2 n2−∑ 4∋+ ∑ 2 i2
i=1 i=1 i=1 i=1 i=1 i=1

n n n
¿ 1+12(n−1+1)+10 n (n−1+1)−6 ∑ i+2 n2 (n−1+1)−4 n ∑ i+2 ∑ i 2
i=1 i=1 i=1

¿ 1+12 n2 +10 n2−6 ( n( n+1

2
)
) +2 n −4 n (
3 n ( n+1 )
2 )+ 2(
n ( n+1 ) (2 n+1)
6
)

3 2
2 2 2 2 2n +3n +n
¿ 1+12 n +10 n −3 n −3 n−2 n +2( )
3

Lecture:4
1. for ¿
cout << “Ali”;
¿ O(n)

2. Nested Loop:
for ¿  (n)
for ¿  (n)
cout << “Ali”;
2
¿ O(n )

for ¿  (n)
  Dependent i

3.

Here,

i 1 2 3 …. n
j 1 time 2 time 3 time …. n times

¿ 1+2+3+…+ n
n ( n+1 )
¿
2
2
¿ O(n )

4.
for ¿  (n)
for ¿  Dependent i
cout << “Ali”;
2
¿ O(n )

Here,

i 1 2 3 …. n
j n time n-2 time n-3 time …. n-n times
 ¿ n+ n−2+ ( n−3 ) +…+ 3+2+1
n ( n+1 )
¿
2
2
¿ O(n )

5.
for ¿  i≤ √ n →( √ n)
cout << “Ali”;
¿ O( √ n)

6.
for ¿ n
for ¿  Dependent i when j<=n
for ¿  100
cout << “Ali”;

Here,

i 1 2 3 …. n
j 1 time 2 time 3 time …. n times
k 1*100 2*100 3*100 …. n*100

¿ ( 1∗100 )+ ( 2∗100 )+ (3∗100 ) + …+( n∗100)

¿ 100(1+2+3+ …+n)
n ( n+1 )
¿ 100( )
2
2
¿ O(n )
7.

for ¿ n
for ¿  Dependent i
for ¿  will always execute n/2
cout << “Ali”;

Here,

i 1 2 3 …. n
j 12 time 22 time 32 time …. n2 times
k 12*(n/2) 22*(n/2) 32*(n/2) …. n2*(n/2)

2∗n 2∗n 2∗n 2∗n

¿(1¿ ¿ )+(2 ¿ ¿ )+(3 ¿ ¿ )+ …+(n ¿ ¿ )¿ ¿ ¿ ¿
2 2 2 2
n 2 2 2
¿ (1+ 2 + 3 + …+n )
2
n n ( n+ 1 ) (2 n+1)
¿ ( )
2 6
4
¿ O(n )

8.
for ¿  n/2
for ¿  n/2
for ¿  n
cout << “Ali”;

n
∗n
2
¿ ∗n=O(n3 )
2

9.

for(∫ i=1 ; i≤ n ; i=i∗2)

cout << “Ali”;

When i=i∗2
Here,

i 1 2 4 8 16 …. n
20 21 22 23 24 …. 2k
k
2 =n
k
¿ log 2 =log n

k log 2=log n where log 2 =1

k =log 2 n

When i=i∗3
Here,

i 1 3 9 27 …. n
30 31 32 33 …. 3k

k
3 =n
k
¿ log 3 =log n
k log3=log n

k =log 3 n

When i=i∗ j
¿ log j (n)

10.

for ¿  n/2
for ¿  n/2

for(∫ k=1 ; k ≤ n; k=k∗2)  log2 n

cout << “Ali”;

 n
∗n
2
¿ ∗log 2(n)=n2 log 2 (n)
2

11.
for ¿  n/2

for(∫ j=1 ; j ≤n ; j= j∗2)  log2 n

for(∫ k=1 ; k ≤ n; k=k∗2)  log2 n

cout << “Ali”;

n n 2
¿ ∗log 2 (n)∗log 2 (n)= (log 2 ( n ))
2 2
 12. Function:

A()
{
k
int n = 22
for (int i=1; i<=n; i++)
{
int j=2;
while (j<n)
{
j = j2;
}
print “ALI”;
}
}

k= 1 2 3

n*2 times n*3 times n*4 times

n=22
k
log n=2
k =log ⁡¿
¿O¿

13.
for ¿  (n)
for ¿
cout << “Ali”;

Here,
 i 1 2 3 …. n
j 1-n time 2-n time 3-n time …. n-n times

n n
¿ n+ + + …+1
2 3
1 1 1
¿ n(1+ + +…+ )
2 3 n
¿ n log n

14.
A()
{
int i=1, s*=I;
while(s<=n)
{
i++;
s=s+i;
print “ALI”;
}
}

Here,

i 1 2 3 4 …. k
j 1 3 6 10 …. n

n=1−k

k (k +1)
n=
2
2
k k
n= + (Ignore)
2 2
2
n=k
k =√ n
 ¿ O( √ n)

Lecture 5:
ln n
log 2 n=
ln 2

For large value of n

log n< √ n< n<nlog n<n < n <2 <n !< n
2 3 n n

Asymptotic Analysis:
How can we map the complexity of Algorithm?
O-Notation:
1. f n=Complexity of your algorihms
2
10 n +5 n

2. gn=Standard Functions

log n< √ n< n , …

2

3. Coefficient c
c . log n
4. n ≥ n0

Two methods:
 By basic definition
 By applying Limits
By applying Limits:
fn
lim =v
n→∞ gn
0≤v ≤∞
f n=Og(n)

Example 1:
2
lim 10 n +5 n
n →∞

n2
2
10 n 5 n
¿ lim ( 2 + 2 )
n→∞ n n
 2
10 n 5 n
¿ lim ( 2
+ 2)
n→∞ n n
lim 5
n→ ∞
¿ lim 10+ ¿
n→∞ n
5
¿ 10+
∞
¿ 10
2 2
10 n +5 n=O(n )

Example 2:
2
lim 10 n +5 n
n →∞
n
2
10 n 5 n
¿ lim ( + )
n→∞ n n
2
10 n 5 n
¿ lim ( + )
n→∞ n n
¿ lim 10 n+ lim 5 ¿
n→∞ n→∞

¿ 10(∞)+5
¿5
2 2
10 n +5 n ≠O (n )

Example 3:
2
lim 10 n +5 n
n →∞

n3
2
10 n 5 n
¿ lim ( 3
+ 3)
n→∞ n n
2
10 n 5 n
¿ lim ( 3
+ 3)
n→∞ n n
lim 10 lim 5
n→ ∞ n →∞
¿ + 2
¿
n n
 ¿0
2 3
10 n +5 n=O(n )

By Basic Definition:
We show that:
Example 1:
2 2
3 n +10 n=O(n )

Consider:
10 ≤ n Where 10 = f(n)

Multiplying by n And n = g(n)

2
10 n ≤ n

Add 3 n2
2 2 2
3 n +10 n ≤3 n + n
2
¿4 n
2 2
3 n +10 n ≤ c . n

Example 2:
2
n≤n

Consider:
Multiplying by 10
2
10 n ≤10 n

Add 3 n2
2 2 2
3 n +10 n ≤3 n +10 n
2 2
3 n +10 n ≤13 n
2 2
3 n +10 n ≤ c . n

For Where c =13

2
n≥n And n0 = 1

By basic definition:
 2 2
3 n +10 n=O(n )

Example 1:
1
¿ lim ¿¿¿
n→∞ n
ln 2
∗1
n
¿
ln 2∗2 n
2 2
1 lg n+n +n=O(¿ n )¿
¿
n∗2 n 2 2
lg n+n +n=Ω(¿ n ) ¿
1 1
¿ =
2n ∞
2

¿ 0+1+0=1

Example 2:
nlg n=O ( n 2)
nlg n
¿ lim 2
n→∞ n
lg n ∞
¿ lim = (undefined)
n→∞ n ∞
ln n
lg n=
ln 2
ln n
ln 2
¿ lim
n→∞ n

ln n
∗1
ln 2
¿ lim
n→∞ n
d
ln n
dn
¿ lim ¿
n→∞ d
¿¿
dn
1
ln 2
n
¿ lim ¿
n→∞ ¿¿
Example 3:
n(n+1)
=O ( n2 )
2

n(n+1)
2
¿ lim 2
n→∞ n
2
n n
+
2 2
¿ lim 2
n→∞ n
1 +1 1 1 1 1
¿ lim ( ¿ )= + = + 0= ¿
n→∞ 2 2 n 2 2∗∞ 2 2

here,
n ( n+1 )
0 ≤ c ≤ ∞= =O(n2)
2
n ( n+1 )
0 ≤ c ≤ ∞= =O(n2)
2

Example 4:
n2 =O ( 2 n)

2
n ∞
¿ lim n
= (undefined )
n→∞ 2 ∞

By applying L. Hopital Rule

d 2
n
dn
¿ lim
n→∞ d
2n
dn
2n ∞
¿ lim = (undefined)
n
n → ∞ 2 ln 2 ∞
By applying derivative:
lim 2
2n n→∞
¿ lim n = n ¿
n → ∞ 2 ln 2 2 ¿ ¿¿

Where ln 2 is the constant value

lim 2
n→∞
∞
2 ¿¿¿

Example 4:
lg ( n ) +n2 +n=O ( n2 )

lg( n) n2 n
¿ lim 2
+ 2+ 2
n→∞ n n n
2
lim n lim n
lg( n) n → ∞ n →∞
¿ lim 2
+ 2
+ 2
n→∞ n n n
∞ 1
¿ +1+¿
∞ ∞
¿ 1+0=1

Lecture 6:
Ω (Omega Notation): Standard Function is below your algo. We talk about best
case.
0 ≤ c ≤ ∞θ

f ( n )=O ¿))
0≤c≤∞

f ( n )=Ω ¿))
0≤c≤∞

f ( n )=θ ¿))
2 2
10 n +5 n=O(13 n )
2 2
10 n +5 n=Ω(9 n )
Example 1:
lg n=O ( n )
lg n
¿ lim
n→∞ n
lg n ∞
¿ lim = (undefined)
n→∞ n ∞
ln n
Since, lg n=
ln 2
ln n
ln 2
¿ lim
n→∞ n
1
ln n
∗1
ln 2
¿ lim
n→∞ n
d
ln n
dn
¿ lim ¿
n→∞ d
¿¿
dn
1
ln 2
n
¿ lim ¿
n→∞ ¿¿
Solving References:

A(n)

n n-1

Decrease And Conquer

T ( 0 )=0
T ( n )=T ( n−1 ) +c

Decrease And Conquer

T ( n )=T ( n−1 ) +nc
T ( 0 )=0

T ( n )=T ( n−1 ) +c
T ( n−1 )=T ( n−2 )+ c
T ( n−2 )=T ( n−3 )+ c
T ( n−3 ) =T ( n−4 )+ c

-
-
-
T ( 3 )=T ( 2 ) +c
T ( 2 ) =T ( 1 ) +c
T ( 1 ) =T ( 0 ) + c
T ( n )=0+ c+ c+ c+ …+c
T ( n )=nc
T ( n )=O(n)
Merge Sort:
When value of start is less than the value of end.
Code:
ms(A,S,E)
{
if(S<E)
mid = (S+E)/2;
ms(A,S,m)
ms(A,mid+1,E)
Merge(A,S,m,E)
}
Merge(A,S,m,E)
{
B[S,E]
i=k=s;
j=mid+1;
while(i<=mid && j<=E)
{
if(A[i]<A[j])
B[K++]=A[i++];
else
B[k++]=A[j++];
while(i<=mid)
B[k++]=A[i++];
while(j<=E)
B[k++]=A[i++];
 for(i=S to E)
A[i]=B[i];
}
}

Example:
0 1 2 3 4 5 6 7
A
9 2 11 3 7 8 15 6
S=0 S=7

2 3 9 11 6 7 8 15

B
2 3 6 7 8 9 11 15
K= S
A
2 3 9 11 6 7 8 15
i= S j=mid

Divide And Conquer:

T ( n )=T ( n2 )+T ( n2 )+cn

For 1st For 2nd Merge
Half Half

T ( n )=2 T ( n2 )+cn
Here, c is cost of reduction (Div + Merge)
Binary Search:
Array should be in sorted form.
B.S (A, S, E, n) if (S<=E)
mid = (S+E)/2
if(n==A[mid])
Print mid;
else if (n<A[mid])
B.S (A, S, mid-1, n)
else
B.S (A, mid+1, E, n)
Divide And Conquer:

T ( n )=T ( n2 )+c
Here n = input size
f(n)
{
if(n==1) f(1)
return 1; f(2)
else f(3)

return n*f(n-1); f(4)

f(5)
}
stack
T ( n )=T ( n−1 ) +1
T ( 1 ) =1

So,
T ( n )=T ( n−1 ) +nc
T ( n−1 )=T ( n−2 )+(n−1) c
 T ( n−2 )=T ( n−3 )+(n−2)c
T ( n−3 ) =T ( n−4 )+(n−3)c

-
-
-
T ( 3 )=T ( 2 ) +3 c
T ( 2 ) =T ( 1 ) +2 c
T ( 1 ) =T ( 0 ) +1 c
T ( n )=nc +c ( n−1 ) + c ( n−2 ) + ( n−3 ) c+ …+3 c+ 2 c+1 c
T ( n )=c ¿
n(n+ 1)
¿c( )
2
2
¿ O(n )

Example:
n(n+1)
=O(n 2)
2
lim n(n+1)
n→ ∞
¿ 2
2n
lim n(n+1)
¿ n→ ∞ 2
2n
lim n+1
n→ ∞
¿
2n
n 1
¿ lim ( + )
n→∞ 2n 2n
1 1
¿ lim ( + )
n→∞ 2 2 n

lim 1 lim 1
n→ ∞ n→ ∞
¿ +
2 2n
 1 1
¿ +
2 2(∞ )
1 1 1
¿ + =
2 ∞ 2

Example:
T ( 0 )=0
T ( n )=T ( n−1 ) +c log n

T ( n )=T ( n−1 ) +clog n

T ( n−1 )=T ( n−2 )+ c log ⁡(n−1)
T ( n−2 )=T ( n−3 )+ c log ⁡(n−2)
T ( n−3 ) =T ( n−4 )+ c log ⁡(n−3)

-
-
-
T ( 3 )=T ( 2 ) +c log 3
T ( 2 ) =T ( 1 ) +c log 2
T ( 1 ) =T ( 0 ) + c log 1
T ( n )=0+ c log n+ c log n−1+c log n−2+ …+c log 3+ c log 2+ c log 1
T ( n )=c log (n∗n−1∗n−2∗…∗3∗2∗1)
T ( n )=c log (n !)

T ( 1 ) =c
T ( n )=2 T ( n /2 ) + c , n>1
 T ( n2 )=2 T ( n4 )+c
T ( )=2 T ( )+c
n n
4 8

T ( n )=2 T ( )+c
n
2

Here c=2 0

[
T ( n )=2 2 T ( n4 )+ c]+ c
¿4 T ( n4 )+3 c
[ () ]
T ( n )=4 2 T
n
8
+ c +3 c

Complexity of Merge Sort:

T ( 1 ) =c
T ( n )=2 T ( n /2 ) + c n
n
T ( n /2 )=2 T ( n /4 ) + c( )
2
n
T ( n /4 )=2 T ( n /8 )+ c ( )
4
T ( n )=2 T ( n /2 ) + cn
T ( n )=2 T ( n /2 ) + cn

[
T ( n )=2 2 T ( n4 )+ c( n2 )]+cn
T ( n )=4 T ( n4 )+ cn+cn
T ( n )=4 T ( )+2 cn
n
4

[
T ( n )=4 2 T ( n8 )+ c ( n4 )]+2 cn
 T ( n )=8 T( n8 )+cn+ 2 cn
T ( n )=8 T ( )+3 cn
n
8

-
-
-

T ( n )=2 T
k
( 2n )+ kcn
k

n
Here, k
=1
2
k
T ( n )=2 T ( 1 ) +kcn
k
n=2 , k=log n
T ( n )=nc +log n . cn
T ( n )=O(n log n)

Example:
1.
T ( n )=2 T ( n /2 ) + cn
Where, a = 2, b = 2 and x = 1
x
ab
1
22
2=2→ θ ¿
θ¿
 2.
3
T ( n )=2 T ( n /2 ) + n
Where, a = 2, b = 2 and x = 3
x
ab
3
22
x
2<8 → θ(n )
3
¿ θ(n )

Master Theorem:
Recurrences:

T ( n )=1 T ( n2 )+ c n 0

Where, a = 1, b = 2 and x = 0
x
ab
0
12
1=1 → θ ¿
θ¿
Best Case:
Quick Sort:

( n2 )+(n−1)
T ( n )=2 T

T ( )=2 T ( )+( −1)

n n n
2 4 2

T ( )=2 T ( )+( −1)

n n n
4 8 4

Here,

T ( n )=2 T ( n2 )+(n−1)
T ( n )=2[2 T ( )+( −1)]+(n−1)
n n
4 2
 2
¿2 T
( ) n
2
2
+ ( n−2 ) +(n−1)

¿2 T
( 2n )+2 n−3
2
2

0 1
¿ 2 +2
k
¿2 T
( )
n
2
k
0 1 k−1
+ kn−(2 +2 +…+2 )

¿ nT (1 )+ ¿
¿ n(0)+ ¿
¿O¿
3.

( n2 )+n
T ( n )=2 T

T ( )=2 T ( )+( )
n n n
2 4 2

T ( )=2 T ( )+( )
n n n
4 8 2

T ( n )=2[2 T ( )+( )]+n

n n
4 2

¿2 T
( 2n )+n+ n
2
2

¿2 T
( 2 )+2 n
n
2
2

Count Sort: (Small No’s)

1 2 3 4 5 6 7 8 9 10 11

9 7 1 2 1 5 9 3 7 2 8 Scan

1 2 3 4 5 6 7 8 9 10 11

C Discard
 0+1+1 0+1+ 0+1 0 0+ 0 0+1+1 0+ 0+1+1 0 0
=2 1 =1 0 1 0 =2 1 =2 0 0
=2 =1 =1

count (A, n, max)

{
c[max] = 0;
for (i=1 to max)
c[max] = 0;
for(i=1 to n)
c[A[i]] = C[A[i]] + 1;
for (i=2 to max)
c[i] = c[i]+c[i-1]
for (i=n down to 1)
B[c[A[i]]] = A[i];
C[A[i]]-=1;

Substitution Method:
Complexity of Binary Search
T ( 1 ) =c

( n2 )+c
T ( n )=T

T ( )=T ( )+c
n n
2 4

T ( )=T ( )+c
n n
4 8
So,

T ( n )=T ( n2 )+c
T ( n )=[T ( )+c ]+c
n
4

¿ T ( ) +2 c
n
4

T ( n )=[T ( )+2 c ]+c

n
8

T ( n )=T ( )+3 c
n
8

-
-
- n
=1
k
2
T ( n )=T
( 2n )+kc ∴ 2n =1
k k
n=2
k

k
¿ T ( 1 ) + kc log ( n )=log 2

¿ c +kc=O(k ) log ( n )=k log 2

¿ O(log (n)) log ( n )=k (1)

k =log ( n )

Example:
Sort the following Array using heap sort?
9 7 11 2 13 5 19 3 17 21 8

7
11

2 8 13

5 19
 1

9
3
2 7
11
7
6
4 2 13
19
5 5

3 17 8
21

8 9 10 11

7
11

2 21
5 19

3 17 8
13

9 3 7 2 1
Iteration 1:

3 9 7 2 1

3 7 9 2 1

3 7 2 9 1

3 7 2 1 9

Iteration 2:

3 7 2 1 9

3 2 7 1 9

3 2 1 7 9

Iteration 3:

2 3 1 7 9

2 1 3 7 9

Iteration 4:

1 2 3 7 9
 k
¿2 T
( 2n )+ kn
k

¿ nT (1 )+ ¿
¿ n(1)+¿
¿ n+¿
¿O¿
Worst Case:
T ( n )=T ( 0 )+ T ( n−1 )+(n−1)
T ( n )=T ( n−1 ) +(n−1)
T ( n−1 )=T ( n−2 )+(n−2)
T ( n−2 )=T ( n−3 )+ ⁡(n−3)

-
-
-
T ( 3 )=T ( 2 ) +2
T ( 2 ) =T ( 1 ) +1
T ( 1 ) =T ( 0 ) +0
T ( n )=( n−1 ) + ( n−2 )+ ( n−3 )+ …+2+1+0
(n−1)(n−1+1)
¿
2
2
T ( n )=O(n )

Bubble Sort:
 for (int i =0; i<n-1; i++)
for (int j=0; j<n-i*2; j++)
{
if (a[j]>a[j+1])
swap(a[j], a[j+1])
}

Insertion Sort:
Insertion Sort (A, n)
{
for (j=2; j<n; j++)
key = A[i];
i = j-1;
while (i>0 && A[i]>key)
A[i+1] =A[i];
i = i-1;
A[i+1] = key;
}
Worst Case:
1+1=2 2(1)
2+2=4 2(2)
3+3=6 2(3)

-
-
-
( n−1 ) +(n−2)=2(n−1)
 2 ( 1 )+ 2 ( 2 )+2 ( 3 ) + …+2(n−1)
2(1+2+3+ …+ ( n−1 ))

¿ ( ( n−1) ( n−1+1
2
)
)=O(n )2

Selection Sort:
Selection Sort(A, n)
{
int min;
for (int i=0; i<=n-1; i++)
min = i;
for (int j=j+1; j<n;j++)
{
if(A[j]<A[min])
{
min = j;
}
if (min != j)
{
swap (A[i]=A[min])
}
}
}
Sequential Sort:
Sequential Sort (A, n)
{
 for(int i=0; i<=n-1;i++)
for(int j=i+1;j<=n;j++)
{
if(A[j]>A[i]
{
temp = A[i];
A[i] = A[j];
A[j] = temp;
}
}
}

Complexity:

i=0 1 2 … n-1
j=n n-1 n-2 … 1

¿ n+ n−1+n−2+…+1
n(n+1)
¿
2
2
¿ O(n )
Graph:
G=(V , E)
V ={a , b , c , d , e , f }
E={( a ,b ) , ( b , c ) , ( c , d ) , ( a , e ) , ( a , f ) }

 Directed:
a i . a j ≠ a j . ai i ≠ j
 Undirected:
a i . a j=a j . ai i ≠ j
 Complete: Every node is connected with every node.

a b c e

d
f

Directed:

a b n ( n−1 )=6
3 ( 3−1 )
c

Undirected:

a b n ( n−1 )
2
4 ( 4−1 )
¿ =6
c d 2
Depth First Search:
DFS (G, S)
For each vertex v ε G(v )
state (v)  Ready
state (s)  wait
push (S, s)
while (S≠ ∅ )
u  pop (S)
for each vertex V ε Adj (u)
if(state(v) = = Ready)
push (S, v)
state (v)  wait
state (u)  processed

F
D
B D E
C
A B
F
A
S
C E

Starting vertex = A
U=AB&C
U=CE
U=ED&F
A C E F D B
Minimum Spanning Tree:
Spanning Tree is weighted always and its edges can be connected in different
ways.
1) Kruskal:
 Sort edges
 Consider each edge (min to max) if do not form cycle.
 Concept of union set.

Example:

7 4
B D F
3 6
1 8
A 5 5 2 H

2 E G 4
C
9 3

Sort:
C, D 1
F, G 2
A, C 2
A, B 3
D, F 4
G, H 4
D, E 5X
B, C 5X
F, H 6X
B, D 7X
E, F 8X
C, E 9X

4
F
 B D
3
1
A 2 H

2 E G 4
C
3

¿ 3+2+1+ 4+3+2+ 4=19

UV ={ { A } , { B } , {C } , { D } , { E } , { F } , { G } , { H } } → C , D ,1

¿ { { A } , { B } , {C , D } , { E } , { F } , { G } , { H } } → F , G ,2

¿ { { A } , { B } , {C , D } , { E } , { F , G } , { H } } → A ,C , 2

¿ { { A , C , D } , { B } , { E } , { F , G } , { H } }→ A , B , 3

¿ { { A , B ,C , D } , { E } , { F , G } , { H } } → E ,G , 3

¿ { { A , B ,C , D } , { E , F , G } , { H } } → D , F , 4

¿ { { A , B ,C , D , E , F , G } , { H } } → G , H , 4

¿ { { A , B ,C , D , E , F , G , H } }

Kruskal Algorithm:
kruskal (G, w)
for each vertex v ∈ V
uv = uv U {v}
for each egde (u, v, w) ∈ E
Enqueue (Q, u, v, w)
while (|uv| > 1)
(u, v, w)  Dequeue (Q)
if u, v belongs to different set us1 and us2 of UV set
uv = uv - us1
uv = uv - us2
uv = uv U (us1 U us2)
T = T U (u, v, w)
return T;

|V| + |E| + |E log E|

Here,
V  UV set
E  Construction of MST
E log E  Sorting

O (|V| + E log E)

Prims:
Prim's Algorithm is a greedy algorithm that is used to find the minimum spanning tree from a
graph.

Algorithm:
Prims (G, s)
T=∅
S = {}
S=SUs
V=V-s
while (v ≠ ∅ )
for each u ε S
for each v ε Adj(u)
 d∞
if (d > u, v, w)
dw
p=u
q=v
S=SUq
U=v-q
T = T U (p, q, d)
return T;
Complexity:
Complexity of Prims Algo is=O(v∗v )

BFS:
BFS (T, P)
n= length(T)
m = length (P)
for (i=0 to n-m)
for (j=1 to m)
if (T (i + j) = P(j))
if (j = m)
return i;
else
brute innerloop
return -1;
Example:
In 1101001. Search 001.
Solution:
 001 Mismatch
1101001

001 Mismatch
1101001

001 1st match remaining

1101001 not match

001 Mismatch
1101001

001 All matches

1101001
Note:
The length of pattern = m = 3
If characters in text are less than of pattern, then there will be no possibility left to
further check.
Finite State Automata:
It is the simplest machine to recognize patterns. The finite automata or finite state
machine is an abstract machine that has five elements or tuples. It has a set of
states and rules for moving from one state to another but it depends upon the
applied input symbol.
A Finite Automata consists of the following:
Q: Finite set of states.
Σ: set of Input Symbols.
q: Initial state.
F: set of Final States.
δ: Transition Function.

Example:

10 10
10
5 5 5
-0 1 2 +3

10

5
In this example:
Q: no of states = 4
Σ: {5, 10}
q: Initial state = - sign
F: set of Final States = + sign.
δ: Transition Table which shows the change from one state to another.
 5 10
0 1 2
1 2 3
2 3 R (∞ )
3 1 2

String Matching:
The problem of finding occurrence(s) of a pattern string within another string or
body of text. There are many different algorithms for efficient searching.
Applications which use string matching is:
 Word Processor (Spell Check, Search/find word)
 Digital Library
 Search Query
 Virus Scanning
 Email Filtering
These points should be kept in mind when string matching:
 Empty String ε “ ”
 Length of the string
 No of characters
 String you want to search = pattern
 In which you want to search = text
 Prefix = Starting character ⊏
 Suffix = Ending character ⊐
There are four algos for string matching:
1. Brute Force Algo
2. Robin Karp
3. FSA based string
4. Knuth Moris Pratt
Graph Traversal:
Graph traversal is a technique used for a searching vertex in a graph. A graph traversal finds the
edges to be used in the search process without creating loops. That means using graph traversal we visit
all the vertices of the graph without getting into looping path.

There are two types of graph traversal which is:

 DFS (Depth First Search) which implements the methodology of Stack.
 BFS (Breadth First Search) which implements the methodology of Queue.
The nodes are:
 Ready (Initial/ unprocessed, before going inside stack)
 Wait (Inside Stack/ Queue)
 Processes (Pop/ Dequeue from Stack/Queue)
Steps:
 Initial State
 Mark all other state as Ready.
 Push/Enqueue the Initial state.
 Change starting state to wait.
 While stack/Queue is not empty
U  Pop(s)
Dequeue (Q)
For each vertex V ε Adj (u)
If (state (v) == Ready)
Enqueue | push (s, v)
State (v)  wait
State (u)  processed

Weighted Graph:
a i . a j=w

Graph Representation:
 Adjacency Matrix
 Adjacency List
Example:
 1 , ai . a j ε E
Ai j={ }
0 ,otherwise

b c

f
a

d e

Adjacency Matrix:
a b c d e f
a 0 1 0 1 0 0
b 0 0 1 1 0 0
c 0 0 0 0 1 0
d 0 0 0 0 1 0
e 0 0 0 0 0 0
f 0 0 0 0 1 0

Adjacency List:

a b d

b c d

c e f

d e
e
 f e

Note:
Less space required in adjacency list than adjacency matrix.

Algo:
FSA (P, ε )
m  length (p)
for (q=0 to m) q: states
for each character e in ε
k  min (m+1, q+2)
Until Prefix Pk matches with suffix q k c. Here
do
q k : previous stage
K = K-1
c :current character
δ ( sigma )( q , c ) ← k

return δ ;

Matcher (T, δ , m)
{
n  length (T)
q = 0; \\ Starting from initial state
for (i=0 to n)  O(n)
c = T [i] \\ Read character from text
q = δ (q ,c ) \\ see transition
if (q = m) \\ if state is final
print found at i-m; \\ starting index of pattern
}
Dynamic Programming:
Dynamic programming is used where we have problems, which can be divided into similar sub-
problems, so that their results can be re-used. Mostly, these algorithms are used for optimization.

Three types of dynamic programming are:

 Decrease and Conquer
 Divide and Conquer (Merge Sort)
 Graph
Divide a problem into overlapped subproblems
 Recurrence Relation
 Recursive Algorithm
 Solve for an Instance
 Memoized Algo
 Pattern, Develop iterative Algo

Example:
Series:
1 , n ≤1
f ( n )={ }
f ( n−1 )+ f ( n−2 ) , otheriwse

f (n)
{
if (n ≤ 1)
return 1
else
return f(n-1) + f(n-2)
} 0 1 2 3 4 5 6

A 1 1 -1 -1 -1 -1 -1

0 1 2 3 4 5 6

A 1 1 2 3 5 8 13
 f s n

n
f s

8+5=13

f (6)
5+3

f (5) f (4 )

3+2
f (4 ) f (3)
2+1

f (3) f (2)

2
f (2) f (1)

1 1

f (1) f (0)

Iterative Algo:
{
A [0] = 1
f=1
A [1] = 1
s=1 for (i= 2 to n)
for (i=2 to n) A[i] = -1;

n=f+s
(n)
Display n;
 f=s
s=n
}

Memoize Algo:
if (n)
{
if (A[n]! = -1)
return A [n];
else
A[n] = f(n-1) + f(n-2)
return A[n];
}

Binomial Co-efficient:

B (nk)= k ! (n−k
n!
)!
1 , n=k∨k=0
B ( n , k )={ }
B ( n−1 , k−1 ) + B ( n−1 , k ) , otherwise

Algo:
B (n, k)
if (n=k or k=0)
return 1
else
return B ( n−1 , k−1 )+ B ( n−1 , k )

0 1 2 3 4 5 6

n
 0 1 i-1, j-1 i-1, j
1 1 1
2 1 2 1
3 1 3 3 1
4 1 4 6 4 1
5 1 -1 10 10 -1 1
6 1 -1 -1 20 -1 -1 1

B(6 , 3)

B(5 , 2) B(5 , 3)

B(4 , 1) B(4 , 3)
B(4 , 2) B(4 , 2)

B(3 , 1) B(3 , 1) B(3 , 2) B(3 , 2) B(3 , 3)

B(3 , 0)

B(2 , 0) B(2 , 1) B(2 , 1) B(2 , 2) B(2 , 2) B(2 , 3)

B(1 , 0) B(1 ,1)

Memoize Algo:
BC (n, k) for (i=0 to n)
if (A[n][k]! = -1) for (j=0 to k)
return A[n][k] if (i=j or j=0)
else A[i][j] = 1

A[n][k] = BC ( n−1 , k −1 ) +BC ( n−1 , k ) else

return A[n][k] A[i][j] = -1;

Example:

A B C
3 5 5 6

1. ( A∗B )∗C
Sol:
A∗B=2∗3∗5=30
( A∗B )∗C=2∗5∗6=60
¿ 30+60=90

2. ( B∗C )∗A
Sol:
2 3
B∗C=3∗5∗6=90

A∗(B∗C)=2∗3∗6=36
¿ 90+36=126

Ways to solve ( A¿¿ 1)( A ¿ ¿ 2 A3 A 4 )¿ ¿

 ( A¿¿ 1)((A ¿ ¿ 2 A 3) A4 ) ¿ ¿
 ( A¿¿ 1)¿ ¿ ¿
 ( A ¿ ¿ 1 A2 )(A 3 A 4 )¿
 ( A ¿ ¿ 1 A2 A 3 )(A 4 )¿
 ((A ¿ ¿ 1 A 2) A 3 )(A 4 )¿
 ( A ¿ ¿ 1( A 2 A 3))(A 4 )¿

A1 … A 4 = A1..1 . A 2..4

¿ A1..2 . A3..4

¿ A1..3 . A 4..4

Example:

A1 A2 A3 A4
 3 4 4 5 4 5

P1 P2 P2 P3 P3 P4

Solution:
1 2 3 4
1 10158 0 24 64 76
2 962 0 60 64
3 37 0 40
4 4 0

2 3
m [ 1. .2 ] =k =1  m [ 1. .1 ] +m [ 2. .2 ] + P i−1 . P k . P j
P0 P1
0+ 0+2.3 .4=24

m [ 2. .3 ]=k=2  m [ 2. .2 ] +m [ 3. .3 ] + Pi−1 . Pk . P j

0+ 0+3.4 .5=60

m [ 3. .4 ] =k =3  m [ 3. .3 ] +m [ 4..4 ] + Pi−1 . Pk . P j

0+ 0+4.2 .5=40

m [ 1. .3 ] =k=1  m [ 1. .1 ] +m [ 2. .3 ] + Pi−1 . Pk . P j

¿ 0+60+ 2.3.5=90

k =2  m [ 1. .2 ] +m [ 3. .3 ] + Pi−1 . Pk . P j

¿ 24+ 0+2.4 .5=64

m [ 2. .4 ] =k =2  m [ 2. .2 ] +m [ 3. .4 ] + Pi−1 . Pk . P j

¿ 0+ 40+3.4 .2=64

k =3  m [ 2. .3 ] +m [ 4..4 ] + Pi−1 . Pk . P j

¿ 60+ 40+3.2 .5=90

Problem:
 A1 A2 A3 A4 A5
A1 10 815 0 120 88 158 160
A2 962 0 48 104 114
A3 73 0 84 78
A4 4 0 42
A5 0

Solution:
m [ 1. .2 ] =k =1  m [ 1. .1 ] +m [ 2. .2 ] + P i−1 . P k . P j

0+ 0+5.4 .6=120

m [ 2. .3 ]=k=2  m [ 2. .2 ] +m [ 3. .3 ] + Pi−1 . Pk . P j

0+ 0+4.6 .2=48

m [ 3. .4 ] =k =3  m [ 3. .3 ] +m [ 4..4 ] + Pi−1 . Pk . P j

0+ 0+6.2.7=84

m [ 4..5 ] =k =4  m [ 4..4 ] +m [ 5. .5 ] + Pi−1 . Pk . P j

0+ 0+2.7 .3=42

m [ 1. .3 ] =k=1  m [ 1. .1 ] +m [ 2. .3 ] + Pi−1 . Pk . P j

0+ 0+5.4 .2=88

m [ 1. .4 ] =k=1  m [ 1. .1 ] +m [ 2. .4 ] + Pi−1 . Pk . P j

0+64 +2.3 .2=76

k =2  m [ 1. .2 ] +m [ 3. .4 ] + Pi−1 . Pk . P j

24+ 40+2.4 .2=80

k =3  m [ 1. .3 ] +m [ 4..4 ] + Pi−1 . Pk . P j

64+ 0+2.5 .4=104

Example:

A2 A3 A4 A5
A1
 4 6 6 2 2 7 7 3

m [ i .. j ] =0 P1 P2 P2 P3 P3 P4 P4 P5

m [ i .. j ] =min i≤ k ≤ j (m [ i .. k ] +m [ k +1. . j ] + Pi−1 . Pk . P j )

k =2  m [ 2. .2 ] +m [ 3. .4 ] + Pi−1 . Pk . P j

¿ 120+84 +5.6 .7=414

k =3  m [ 1. .3 ] +m [ 4..4 ] + Pi−1 . Pk . P j

¿ 88+ 0+5.2.7=158

5 4 k =2  m [ 2. .2 ] +m [ 3. .5 ] + Pi−1 . Pk . P j

¿ 0+78+ 4.6 .3=150

P0 P1
k =3  m [ 2. .3 ] +m [ 4..5 ] + Pi −1 . P k . P j

¿ 48+ 42+ 4.2.3=114

k =4  m [ 2. .4 ] + m [ 5..5 ] + Pi−1 . P k . P j

¿ 104+ 0+4.7 .3=188

Algo:
m [i, j]  ∞
for k=1, j-1
do t  m [i, k] + m [k+1, j] + Pi−1 . P k . P j
if (t < m [i, j])
then m [i, j]  y, s [i, j]  k

Multiply (i, j)
if(i=j)
then return A[i]
else k  s [i, j]
X  Multiply [i, k]
Y  Multiply [k+1, j]
return X.Y

(1 , 5)
(1 , 5)
3

(1 , 3) (4 , 5)
1 4

A1 A5 (1 , 1) (2 , 3) (4 , 4) (5 , 5)
2 A4
A4 A5

A2 A3
A1 (2 , 2) (3 , 2)

Longest Common Subsequence: A2 A3

A [i, j] = A [i-1, j-1] + 1, S1 [i] = S2 [j]

A [i, j] = max A [i-1, j], A [i, j-1] otherwise
S1 = A C D X Y
S2 = A G X D F
A C D X Y
0 0 0 0 0 0
A 0 1 1 1 1 1
G 0 1 1 1 1 1
X 0 1 1 1 2 2
D 0 1 1 2 2 2
F 0 1 1 2 2 2
BT (i, j)
while (i>0 || j>0)
{
if (A [i, j] = A [i-1, j-1] + 1)
S = S1 [i] U S
i--;
j--;
else
if (A [i, j] = A [i, j-1])
j--;
else
i--;
}

EDIT Distance:

NULL F R E Q U E N T
NULL 0 1 2 3 4 5 6 7 8
F 1 1 2 3 4 5 6 7 8
R 2 2 2 2 3 4 5 6 7
E 3 3 3 3 2 3 4 5 6
Q 4 4 4 4 3 2 3 4 5
U 5 5 5 4 4 3 2 3 4
E 6 6 6 5 5 4 3 2 3
N 7 7 7 6 6 5 4 3 3
T 8 8 8 7 7 6 5 4 4

Tower of Hanai:
 empty

A D

D A

TOH (n)
if (n = 1)
move (n, S, D, A)
else
move (n-1, S, A, D)
move (n, S, D, A)
move (n-1, A, D, S)

Knap Sack Problem:

items=i 1 ,i 2 , i3 , … , i n
values=v 1 , v 2 , v 3 , … , v n
weight =w1 , w2 , w3 , … , w n
weight of KS=W

Fractional
KSP
 KS
0/1 KSP

Huffman Coding:

Character Frequency Fixed-length Variable

codeword
P 30 000 01
Q 25 001 11
R 20 101 000
S 15 011 001
T 13 100 101
U 10 101 1000
V 5 110 1001

Algo:
Huffman (c )
n  |c|  set of n characters
Q  c  min priority queue
for i =1 to n
do
n-1  allocate a new node
left [z]  x  Extract-Min(Q)
Right [z]  y  Extract-Min(Q)
f[z]  f [x] + f[y]
insert (Q, z)
return Extract-Min(Q)
 LCS Algo:
LCS (S1, S2)
for (i=0 to length(S1))
A [0, i] = 0
for (j=0 to length(S2))
A [i, 0]  0
for (i=0 to length(S1))
for (j=0 to length(S2))
if (S1[i] = S2[j])
A [i, j] = A [i-1, j-1] + 1
else
A [i, j] = max ([i-1, j] , A [i, j-1])

Matrix Chain Algo:

Matrix Chain (p, N)
For i=1 to N
do m [i, i]  0
For L = 2, N
do
For i=1, n-L+1
do j  i+L-1
If a chain of matrices is given, we have to find the minimum number of the correct
sequence of matrices to multiply.
We know that the matrix multiplication is associative, so four matrices ABCD, we
can multiply A(BCD), (AB)(CD), (ABC)D, A(BC)D, in these sequences. Like
these sequences, our task is to find which ordering is efficient to multiply.
In the given input there is an array say array, which contains array[] = {1, 2, 3, 4}.
It means the matrices are of the order (1 x 2), (2 x 3), (3 x 4).

Example:
Input:
The orders of the input matrices. {1, 2, 3, 4}. It means the matrices are
{(1 x 2), (2 x 3), (3 x 4)}.
Output:
Minimum number of operations need multiply these three matrices. Here the result
is 18.