Unit –I

Introduction to Algorithm
Amrita Naik
 Algorithm
• An algorithm is finite set of instruction if
followed accomplishes a particular task.
• Ex:
Algorithm checkEven(n)
{
If(n%2==0)
Print “Even”
Else
Print”Odd”
}
 Algorithm Example
• Step 1: Take the brush
• Step 2: Put the paste on it.
• Step 3: Start brushing
• Step 4: Rinse
• Step 5: Wash
• Step 6: Stop
 Criteria to design efficient Algorithm
• 1.Input
• 2.Output
• 3.Definiteness
• 4.Finiteness
• 5.Effectiveness
 Performance Analysis
• 1.Space Complexity: is the amount of memory
it needs to run to completion.
– Represented by S(P)
• 2.Time Complexity: is the amount of computer
time it needs to run to completion.
– Represented by T(P)
 Space Complexity(S(P))
• The space needed by each algorithm is sum of:
• (i)Fixed part: Space for code, fixed variable
and constants - C
• (II)Variable part: Space needed by reference
variable ,recursion stack - Sp
• S(P)= C+Sp
 Example1 for space complexity

Algorithm abc(a,b,c)
{
return a+b+b*c+(a+b-c)/(a+b)
}

• C=3
• Sp=0
• S(P)=C+Sp=3+0=3
 Example2 for space complexity
Algorithm sum(a,n)
{
s=0
for i=1 to n do
s=s+a[i]
Return s
}
• C=3
• Sp=n
• S(P)=C+Sp=3+n
 Example3 for space complexity
Algorithm RSum(a,n)
{
If(n<=0) then
return 0;
Else
return Rsum(a,n-1)+a[n];
}
• C=0
• Sp=3(n+1)
• S(P)=0+Sp=0+3(n+1)
 Time Complexity
• 2 methods:
• 1)using Global variable
• 2)using Tabular method
1)Calculate Time complexity using Count variable
Algorithm sum(a,n)
{
s=0;
count=count+1
for i=1 to n do
{
count=count+1
s=s+a[i] T(n)=No of times
count=count+1 count is
} incremented=1+n
count=count+1 n+1+1
count=count+1
Return s T(n)=2n+3
}
 2)Using tabular method
s/e frequency Total
steps
• Algorithm sum(a,n) 0 - 0
• { 0 - 0
• s=0 1 1 1
• for i=1 to n do 1 n+1 n+1
• s=s+a[i] 1 n n
• return s 1 1 1
• }
0 - 0

2n+3
 Another Example
s/e F(n=0) F(n> TS(n=0 TS(n>0
0) ) )

0 - - 0 0
Algorithm Rsum(a,n)
{ 0 - - 0 0
If (n<=0) then 1 1 1 1 1
return 0 1 1 0 1 0
Else 0 - - - -
return Rsum(a,n-10)+a[n] 1+X 0 1 0 1+X
} 0 - 0

2 2+X

T(n)=2+2+2…n+1 times=2(n+1)
 Another Example
s/e Frequency Total
count
0 - 0
Algorithm Add(a,b,c,m,n)
{ 0 - 0
for i=1 to m do 1 m+1 m+1
for j=1 to n do 1 m(n+1) mn+m
c[i,j]=a[I,j]+b[I,j] 1 mn mn
} 0 - 0
2mn+2m
+1

T(n)=2mn+2m+1
An Example of Linear Search Algorithm
• Start from the leftmost element of array and
one by one compare x with each element of
array
• If x matches with an element, return the
index.
• If x doesn’t match with any of elements,
return -1.
• Ex: Consider an array
A={10,50,30,70,80,60,20,90,40}
 Linear Search

The best time complexity of linear search is O(1)

The average time complexity of the above algorithm
is O(n)
The worst time complexity of the above algorithm is
O(n)
 Comparison of growth function

Example of Time Complexity: T(n)=n2+2n+3

 Asymptotic Notations
• 1.Big Oh(O) : “f(n) is O(g(n))” iff for some
constants c and N₀, f(n) ≤ cg(n) for all N > N₀
• 2.Big Omega(Ω):f(n)= Ω (g(n)), iff for some
constants c and N₀, f(n) ≥ cg(n) for all N > N₀
• 3.Big Theta(Ө):iff f(n) is O(g(n)) and f(n) is Ω
(g(n))
1.Big Oh(O)
 Big Oh Example
• Example: P.T 2n + 10 is O(n)
▪ 2n + 10 ≤ cn
▪ 2+10/n ≤ c [Divide both sides by n]
▪ Pick c >=12 and n0 = 1
Big-Omega
■ f(n) is Ω(g(n)) if there is a constant c > 0
and an integer constant n0 ≥ 1 such that
f(n) ≥ c•g(n) for n ≥ n0

Analysis of Algorithms 21
 Big-Theta
❑f(n) is Θ(g(n)) if there are constants c’ > 0
and c’’ > 0 and an integer constant n0 ≥ 1
such that c’•g(n) ≤ f(n) ≤ c’’•g(n) for n ≥ n0

Analysis of Algorithms 22
 Different Type of Algorithm
• 1.Divide and Conquer
• 2.Greedy
• 3.Dynamic
• 4.Backtracking
• 5.Branch and Bound
 1.Divide and Conquer
• 1. Divide the problem into two or more
smaller subproblems.
• 2. Conquer the subproblems by solving
them recursively.
• 3. Combine the solutions to the
subproblems into the solutions for the
original problem.
 General Method
Divide_Conquer(problem P)
{
if Small(P)
return S(P);
else
{
Divide P into smaller instances P1, P2, …,
Pk;
Apply Divide_Conquer to each of these
subproblems;
Return Combine(Divide_Conque(P1),
Divide_Conque(P2),…, Divide_Conque(Pk));
}
}
 1.Binary Search
• Fastest searching algorithm which uses Divide
and Conquer approach.
• Search a sorted array by repeatedly dividing
the search interval in half.
• If the value of the search key is less than the
item in the middle of the interval, narrow the
interval to the lower half.
• Otherwise narrow it to the upper half.
• Repeatedly check until the value is found or
the interval is empty.
Algorithm BinarySearch(a, n,x)
{
low=1, high=n
while(low<=high){
mid=L(low+high)/2 ˩
if(x<a[mid])
high=mid-1
else
{
If(x>a[mid])
low=mid+1
else
return mid
}
}
return “Not Found”
}
 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo hi

• Ex. Binary search for 33 in an array

A={6,13,14,25,33,43,53,64,72,84,93,95,96,97}
 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo mi hi
d
• Ex. Binary search for 33.
 Binary Search
• Binary search. Given value and sorted array a[],
find index I such that a[i] = value, or report that
no such index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo hi

• Ex. Binary search for 33.

 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo mi hi
d

• Ex. Binary search for 33.

 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo hi
• Ex. Binary search for 33.
 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo mi hi
d
• Ex. Binary search for 33.
 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo
hi
• Ex. Binary search for 33.
 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo
hi
mi
d
• Ex. Binary search for 33.
 Binary Search
• Binary search. Given value and sorted array a[],
find index i
such that a[i] = value, or report that no such
index exists.
• Invariant. Algorithm maintains a[lo] ≤ value ≤
a[hi].

6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

lo
hi
mi
d
• Ex. Binary search for 33.
 Time Complexity –Binary Search
• T(n)=T(n/2)+f(n)
• =T(n/2)+c
• [Base step: T(1)=c]
 Recurrence Relation
• Is an equation that recursively describes a time
complexity function for an algorithm given a
base step.
• 4 ways to solve recurrence relation:
– 1)Substitution
– 2)Iteration
– 3)Recursion tree
– 4)Master’s Theorem
 1)Substitution
• 2 Steps:
• 1)We make a guess for the solution
• 2)We use mathematical induction to prove
the guess is correct or incorrect.
Time Complexity of Binary Search using
Substitution
• T(n) = T(n/2) + c
• T(n)=aT(n/b)+f(n)
– We guess the solution as T(n) = O(Logn).
– Now we use induction to prove our guess.
– We need to prove that T(n) <= cLogn.
– We can assume that it is true for values smaller
than n.
T(n) = T(n/2) + c
= T(n/2i)+ ic = cLog(n/2i) + ic
= cLogn - cLog2i + ic
= cLogn - ci + ic =clogn
<= cLogn
 Example of Substitution
• T(n) = 2T(n/2) + n
– We guess the solution as T(n) = O(nLogn).
– Now we use induction to prove our guess.
– We need to prove that T(n) <= cnLogn.
– We can assume that it is true for values smaller
than n.

T(n) = 2T(n/2) + n
= 2c(n/2)Log(n/2) + n
= cnLogn - cnLog2 + n
= cnLogn - cn + n = cnLogn – n(c-1)
<= cnLogn
 2)Iteration Method
T(n)= T(n/2)+c Time Complexity for Binary
T(n)=T(n/4)+c+c Search
=T(n/8)+c+c+c
=T(n/16)+c+c+c+c
=T(n/32)+5c
=T(n/25)+5c
=T(n/2i)+ic
=(c+clogn) [when n=2i ,then log n=i]
T(n)>=clogn
T(n)=Ω(logn)
 3)Recursion Method
Time Complexity for Binary n c
Search
n/2 x c

n/4 x x x c

n/8 x The no of steps taken log n

=clogn time
T(n)=O(logn)
 4)Master’s Theorem
• Direct way of finding the solution , when the
time complexity is following form:
T(n) = aT(n/b) + f(n) where a >= 1 and b > 1
Here f(n) is of the form nd
 Time Complexity for Binary Search
using Master’s Theorem
• T(n) = T(n/2) + c
Here a=1, b=2, d=0
a=bd
Therefore T(n)=O(ndlogn)
=O(logn)
 2)Another Example of Iteration
• T (n) = 1 if n=1
= 2 T (n-1) if n>1

T(n)=2 *(2*T(n-2))
=2*(2*(2*T(n-3)))
=2KT(n-k)
n-k=1 =>k=n-1
=2n-1 T(1)= 2n-1
 2)Another Example of Iteration
• T (n) = 2T (n-1)
= 2[2T (n-2)]
= 22T (n-2)
= 4[2T (n-3)]
= 23T (n-3)
= 8[2T (n-4)]
= 24T (n-4) ------------------------- (Eq.1)
Repeat the procedure for i times T (n) = 2i T (n-i)
Put n-i=1 or i= n-1 in (Eq.1)
T (n) = 2n-1 T (1) = 2n-1 .1 {T (1) =1 .....given}
= 2n-1
 2)Another Example of Iteration
• T (n) = T (n-1) +1 and T (1) = θ (1).
• =T(n-2)+1+1
• =T(n-3)+3
• =T(n-k)+k
• Substitute n-k=1, k=n-1,
• =T(1)+n-1=1+n-1
• =n
 2)Another Example of Iteration
• T (n) = T (n-1) +1 and T (1) = θ (1).
• Solution: T (n) = T (n-1) +1
= (T (n-2) +1) +1
= (T (n-3) +1) +1+1
= T (n-4) +4
= T (n-5) +1+4
= T (n-5) +5
= T (n-k) + k Where k = n-1
T (n-k) = T (1) = θ (1)
T (n) = θ (1) + (n-1) = 1+n-1=n= θ (n).
 3)Another Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:

L2.50
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
T(n/4) T(n/2)

L2.51
 Example of recursion tree

Solve T(n) = T(n/4) + T(n/2) + n2:

n2
(n/4)2 (n/2)2

T(n/16) T(n/8) T(n/8) T(n/4)

 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

Θ(1)

L2.53
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

Θ(1)

L2.54
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2

(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

Θ(1)

L2.55
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:

n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…

…
Θ(1)

L2.56
 Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2

(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

…
…

Θ(1) Total
== Θ(n2) geometric series
L2.57
 4)Another Example of Master’s
Theorem
• (1)T(n) = 3T(n/2) + n2
• (2) T(n)=4T (n/2) + n
4)Another Example of Master’s Theorem
1)T(n) = 3T(n/2) + n2
• Solution: a=3, b=2, d=2
3<22 => a<bd
Therefore T(n)=O(nd)
=O(n2)
2) T(n)=4T (n/2) + n
• Solution:a=4, b=2, d=1
4>21 => a>bd
Therefore T(n)=O(n(log b a))
=O(n2)
 Min -Max
Algorithm MinMax(i,j,max,min)
{ Else
If(i=j) {
return max=min=a[i] mid=L(i+j)/2 ˩
else if(i=j-1)
MinMax(i,mid,max,
{
If(a[i]<a[j]) min)
{ MinMax(mid+1,j,ma
max=a[j] x1,min1)
min=a[i] If(max<max1)
} max=max1
else
{ If(min>min1)
max=a[i] min=min1
min=a[j] }
} }
}
 Ex:{21,6,35,42,-3,18,48,16,7,10,28,100,
41}
1,13,100,-3

1,7,48,-3 8,13,100,7

1,4,42,6 5,7,48,-3 8,10,16,7 11,13,100,28

1,2,21 3,4,42 5,6,18 7,7,48 8,9,16 10,10, 11,12, 13,13,

,6 ,35 ,-3 ,48 ,7 10,10 100,28 41,41
 Min Max Time Complexity
• T(n)= T(n/2)+T(n/2)+2 n>2
1 n=2
0 n=1
• T(n)=2T(n/2)+2
• =2(2T(n/4)+2)+2
• =4T(n/4)+4+2
• =2k-1T(2)+2k-2
• =(2k /2)+2k-2 [n=2k]
• =n/2+n-2
• =3n/2 -2=O(n)
 Merge Sort
• Uses Divide and Conquer strategy
Merge Sort
Merge Sort

h-> j->

1 2 3 4 5

A[low] A[mid] A[high]

B[i]
 Merge sort Example
• Ex:{310,285,179,652,351,423,861,254,450,520
}
Merge Sort Example
Merge Sort Example
 Merge Sort Example
• 310,285,179,652,351,423,861,254,450,520
• 285,310,179,652,351,423,861,254,450,520
• 179,285,310,652,351,423,861,254,450,520
• 179,285,310,351,652,423,861,254,450,520
• 179,285,310,351,652,423,861,254,450,520
• 179,285,310,351,652,423,861,254,450,520
• 179,285,310,351,652,254,423,861,450,520
• 179,285,310,351,652,254,423,861,450,520
• 179,285,310,351,652,254,423,450,520,861
• 179,254,285,310,351,423,450,520,652,861
 Merge Sort Time Complexity
• T(n)= 2T(n/2)+cn n>1, c-constant
a n=1, a-constant
• T(n)= 2T(n/2)+cn
=2(2T(n/4)+cn/2)+cn
=2(4T(n/8)+cn/4)+2cn
=8T(n/8)+3cn
=2kT(1)+kcn
=an+ nclogn [n=2k]
=O(nlogn)
 Quick Sort
• Uses divide and conquer.
• It finds the element called pivot which divides
the array into two halves.
• Elements in the left half are smaller than pivot
and elements in the right half are greater than
pivot.
• Three steps
– Find pivot that divides the array into two halves.
– Quick sort the left half.
– Quick sort the right half.
Quick Sort
Quick Sort
 Example
• data={40,20,10,80,60,50,7,30,100}

pivot_index = 0 40 20 10 80 60 50 7 30 100

[1] [2] [3] [4] [5] [6] [7] [8] [9]

i j
 1. While data[i] <= data[pivot]
++i

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] >= data[pivot]
--j

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.
5. Swap data[j] and data[pivot_index]

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 1. While data[i] <= data[pivot]
++i
2. While data[j] > data[pivot]
--j
3. If i < j
swap data[i] and data[j]
4. While j > i, go to 1.
5. Swap data[j] and data[pivot_index]

pivot_index = 5 7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

i j
 Partition Result

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

Recursion: Quicksort Sub-arrays

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

 Example
• A={65,70,75,80,85,60,55,50,45}
Quick Sort
Quick Sort
Solution
Solution
Solution
Solution
 Time Complexity
• Best case running time: Ω(n log2n)
– T(n)=T(n/2)+T(n/2)+n
• Worst case running time: O(n2)
 Example of Worst Case Time
Complexity
• Quick sort has worst time complexity when
the elements in the list are already sorted.

10
pivot_index = 0 2 4 10 12 13 50 57 63
[1] [2] [3] [4] [5] [6] [ 7] [8]
0
[9]

too_big_index too_small_index
 Example of Worst Case Time
Complexity
T(n)=n+(n-1)+(n-2)+….+2=n(n+1)/2=O(n2)

10
pivot_index = 0 2 4 10 12 13 50 57 63
[1] [2] [3] [4] [5] [6]
0
[7] [8] [9]

<= data[pivot] > data[pivot]

Selection: Finding kth smallest element
• Select a pivot and partition the array with
pivot at correct position j
• If position of pivot, j, is equal to k, return A[j].
• If j is less than k, discard array from start to j,
and look for (k-j)th smallest element in right
sub array, go to step 1.
• If j is greater than k, discard array
from j to end and look for kth element in left
subarray, go to step 1
Selection: Finding kth smallest element

A[j]
1 2 j 4 5
Selection: Finding kth smallest element
• Ex:{12,8,10,14,16,19,3}
• Find second smallest element in this array.
Selection: Finding kth smallest element
Selection: Finding kth smallest element
Selection: Finding kth smallest element
 Finding kth Smallest Element Time
Complexity
• Time complexity of finding kth smallest
element in list is O(nlogn)
• The worst case time complexity of the above
solution is still O(n2).
 Strassen’s Matrix Multiplication
• Naïve Matrix Mutiplication
for (int i = 0; i < N; i++)
{
for (int j = 0; j < N; j++)
{
C[i][j] = 0;
for (int k = 0; k < N; k++)
{
C[i][j] += A[i][k]*B[k][j];
}
}
}
Divide and Conquer Matrix Multiplication
• Following is simple Divide and Conquer
method to multiply two square matrices:
1) Divide matrices A and B in 4 sub-matrices of
size N/2 x N/2 as shown in the below diagram.
2) Calculate following values recursively.
ae + bg, af + bh, ce + dg and cf + dh.

T(N) = 8T(N/2)
+ O(N2)
8-Multiplication
N2-Addition
 Strassen’s Matrix Multiplication

Time Complexity= T(n)=7T(N/2) + O(N2)

 Strassen’s Time Complexity
• T(N) = 7T(N/2) + O(N2)
• From Master's Theorem,
• a=7, b=2,d=2
• a>bd , Hence T(n)=N log b a
Log7 2.8074
• O(N ) which is approximately O(N )
Performance Enhancement with
Strassens
Strassen’s Matrix Multiplication Example

• 1.Perform matrix multiplication using

Strassen’s method.

• A= 2 3
3 2

• B= 4 3
3 4
Strassen’s Matrix Multiplication Example
Strassen’s Matrix Multiplication Example
 Strassen’s Matrix Multiplication
Example
• 1)Solve matrix multiplication using Strassens
multiplication
2 1 1 3
A= 6 0 4 5
3 2 1 8
4 3 2 1

1 0 0 3
B=
2 1 1 6
7 1 8 3
0 5 4 0
Questions Asked
Questions Asked
Questions Asked
Questions Asked
Questions Asked
Questions Asked
 Text Books
• 1 Fundamentals of Computer Algorithms – E.
Horowitz et al, 2nd Edition UP.- Divide and
Conquer Strategy
• 2. Introduction to Algorithms, 3th Edition,
Thomas H Cormen, Charles E Lieserson,
Ronald L Rivest and Clifford Stein, MIT
Press/McGraw-Hill.- Recurrence Relation
• 3. Design and Analysis of Algorithm –V. V
Muniswamy- Problems on Recurrence
Relation(Reference Book)
Thank You
Some Additional Problems
Some Additional Problems
 Hierarchy of functions

1, log2n, 3√n, √n, n, nlog2n, n√n, n2, n3, 2n, n!, nn

Each one is Big-Oh of any function to its right

 Basic Rules of Logarithms
a
If logz (x) = a, then x = z
logz (xy) = logz (x) + logz (y)
logz (x/y) = logz (x) - logz (y)
y
logz (x ) = ylogz (x)
If x = y then logz (x) = logz (y)
If x < y then logz (x) < logz (y)
logz (-|x|) is undefined
Some Equations
Some Equations