DESIGN AND ANALYSIS OF ALGORITHMS

Dr. Kashif Ayyub

1
Binary Search Method

BINARY-SEARCH (A, lo, hi, x)

if (lo > hi) constant time: c1
return FALSE
mid  (lo+hi)/2 constant time: c2

if x = A[mid]
return TRUE constant time: c3

if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x) same problem of size n/2
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x) same problem of size n/2
Binary Search Example

target is 33
The array a looks like this:
Indices 0 1 2 3 4 5 6 7 8 9
Contents 5 7 9 13 32 33 42 54 56 88

mid = (0 + 9) / 2 (which is 4)
33 > a[mid] (that is, 33 > a[4])
So, if 33 is in the array, then 33 is one of:
5 6 7 8 9
33 42 54 56 88

Eliminated half of the remaining elements from

consideration because array elements are sorted.
Binary Search Example
target is 33
The array a looks like this:
Indexes 0 1 2 3 4 5 6 7 8 9
Contents 5 7 9 13 32 33 42 54 56 88

mid = (5 + 9) / 2 (which is 7)
33 < a[mid] (that is, 33 < a[7]) Eliminate
So, if 33 is in the array, then 33 is one of: half of the
5 6 remaining
33 42 elements

mid = (5 + 6) / 2 (which is 5)
33 == a[mid]
So we found 33 at index 5:
5
33
Worst-case Analysis
• Item not in the array (size N)
• T(N) = number of comparisons with array elements
• T(1) = 1
• T(N) = 1 + T(N / 2)
Worst-case Analysis
• Item not in the array (size N)
• T(N) = number of comparisons with array elements
• T(1) = 1
• T(N) = 1 + T(N / 2)
= 1 + [1 + T(N / 4)]
Worst-case Analysis
• Item not in the array (size N)
• T(N) = number of comparisons with array elements
• T(1) = 1
• T(N) = 1 + T(N / 2)
= 1 + [1 + T(N / 4)]
= 2 + T(N / 4)
= 2 + [1 + T(N / 8)]
Worst-case Analysis
• Item not in the array (size N)
• T(N) = number of comparisons with array elements
• T(1) = 1
• T(N) = 1 + T(N / 2) 
= 1 + [1 + T(N / 4)]
= 2 + T(N / 4) 
= 2 + [1 + T(N / 8)]
= 3 + T(N / 8) 
=…
Worst-case Analysis
• Item not in the array (size N)
• T(N) = number of comparisons with array elements
• T(1) = 1
• T(N) = 1 + T(N / 2) 
= 1 + [1 + T(N / 4)]
= 2 + T(N / 4) 
= 2 + [1 + T(N / 8)]
= 3 + T(N / 8) 
=…
= k + T(N / 2k ) [1]
Worst-case Analysis
• T(N) = k + T(N / 2k ) [1]
• T(N / 2k ) gets smaller until the base case: T(1)
• 2k = N
• k = log2N
• Replace terms with k in [1]:
T(N) = log2N + T(N / N)
= log2N + T(1)
= log2N + 1
• “log2N” algorithm
• We used recurrence equations
Recurrence
• A recurrence is an equation or inequality that
describes a function in terms of itself by using
smaller inputs.
• Recurrences arise when an algorithm contains
recursive calls to itself.  c n 1

• What is the actual running time of the T (n) 
algorithm?
2T  n   cn n  1
• Need to solve the recurrence   2 
• Find an explicit formula of the expression
The expression is a recurrence
• Bound the recurrence by an expression that
involves n
Example

Function(int number)
if n<=1
then return;
else
Function(number/2)
Function(number/2)
for(i=1 to number )
Display i;

It follows that work done

T(n) = 2 T(n/2)+ cn dividing and
combining

number of sub- Sub-problem size

problems
Recursion Tree for Algorithm

cn

T(n/2) T(n/2)
Recursion Tree for Algorithm

cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

Recursion Tree for Algorithm

cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

Eventually, the input size (the argument of T) goes to 1, so...

Recursion Tree for Algorithm

cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

T(1) T(1) T(1) T(1) ............................. T(1) T(1)

n sub problems of size 1, but T(1) = c

by boundary condition
Recursion Tree for Algorithm

cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

c c c c ............................. c c

n subproblems of size 1
Recursion Tree for Algorithm
level nodes/ cost/
level level
0 20 = 1 cn
1 21 = 2 cn
2 22 = 4 cn
. .
. .
. .
N-1 2N-1=n cn
Since 2N-1 = n, lg(2N-1) = lg(n)
levels = N = 1+lg(n)
T(n) = total cost = (levels)(cost/level) n nodes at level N-1
T(n) = cn [1+lg(n)] = O( n lg(n))
Methods for Solving Recurrence Relations
• No universal method exists that would enable us to solve every recurrence
relation.
• Following are some that can solve a variety of recurrences.
• Iteration method
• Master method

19
The Iteration Method
• Expand the recurrence k times
• Work some algebra to express as a summation
• Evaluate the summation
Example - 1
• T(n) = c + T(n-1) = c + c + T(n-2)
= 2c + T(n-2) = 2c + c + T(n-3)
= 3c + T(n-3) … kc + T(n-k)
= ck + T(n-k)
• So far for n  k we have
T(n) = ck + T(n-k)

• To stop the recursion, we should have

n-k=0  k=n
T(n) = cn + T(0) = cn  0 n 0
T (n) 
c  T (n  1) n  0
• Thus in general T(n) = O(n)

21
Example - 2
• T(n) = n + T(n-1) Algo A(int n)
= n + n-1 + T(n-2) {
if (n>0)
= n + n-1 + n-2 + T(n-3) {
= n + n-1 + n-2 + n-3 + T(n-4)
for(i=0;i<n;i++)
=…
= n + n-1 + n-2 + n-3 + … + (n-k+1) + T(n-k) printf("%d",n);
}
A(n-1);
• To stop the recursion, we should have n - k = 0  k = n }
 0 n 0
n n
n 1 T (n) 
i 1
i  T ( 0)   i  0  n
i 1 2 n  T (n  1) n  0
n 1
T ( n)  n O(n 2 )
2
22
Example - 3
• T(n) = 2 T(n/2) + c 1
= 2(2 T(n/2/2) + c) + c 2
= 22 T(n/22) + 2c + c
= 22(2 T(n/22/2) + c) + (22-1)c 3
= 23 T(n/23) + 4c + 3c
= 23 T(n/23) + (23-1)c
= 23(2 T(n/23/2) + c) + 7c 4
= 24 T(n/24) + (24-1)c  c n 1
  n
= … T (n) 2T
   c n 1
= 2k T(n/2k) + (2k - 1)c k   2 

23
Example - 3
• So far for n  k we have
• T(n) = 2k T(n/2k) + (2k - 1)c

• To stop the recursion, we should have

n/2k = 1  k = lg n
T(n) = 2lgn T(n/2lgn) + (2lgn - 1)c
= n T(n/n) + (n - 1)c
= n T(1) + (n-1)c
= nc + (n-1)c  c n 1
  n
= nc + nc – c = 2cn – c = cn – c/2 T (n) 2T
   c n 1
 cn = O(n) for all n  ½   2 

24
Example - 4

25
The Master Theorem

26
The Master Theorem
• Given: a divide and conquer algorithm
• An algorithm that divides the problem of size n into a
subproblems, each of size n/b
• Let the cost of each stage (i.e., the work to divide the problem +
combine solved subproblems) be described by the function f(n)

• Then, the Master Theorem gives us a cookbook for the

algorithm’s running time:

27
The Master Theorem
• if T(n) = aT(n/b) + f(n) where a ≥ 1 & b > 1
• then  
 
 logb a
n  
f (n) O n 
log b a   

 
 logb a   0
T (n)  n lg n  
f (n)  n 
log b a

  c 1
 
 f (n)  f (n)  n  log b a 
& 
 
 af (n / b)  cf (n) for large n
28
Using The Master Method: Case 1
• T(n) = 9T(n/3) + n
• a = 9, b = 3, f(n) = n
• nlog a = nlog 9 = n2
b 3

• since f(n) = O(nlog 9 - ) = O(n2-0.5) = O(n1.5)

3

• where  = 0.5
• case 1 applies:

• Thus the solution is T (n)  n logb a   

when f (n) O n logb a   
• T(n) = (n2)
Using The Master Method: Case 2
• T(n) = T(2n/3) + 1
• a = 1, b = 3/2, f(n) = 1
• nlog a = nlog 1 = n0 = 1
b 3/2

• since f(n) = (nlog a) = (1)

b

• case 2 applies:


• Thus the solution is T (n)  n logb a lg n  
when f (n)  n logb a 
• T(n) = (lg n)
  
 logb a 
Using The Master Method: Case T (n) 3 n  lg n  
f (n)  n 
log b a

 c
• T(n) = 3T(n/4) + n lg n  
• a = 3, b = 4, f(n) = n lg n
 f (n)  f (n)  n  
log b a 
& 
 
• nlog a = nlog 3 = n0.793 = n0.8
b 4  af (n / b)  cf (n) for large n
• Since f(n) = W(nlog 3+) = W(n0.8+0.2) = W(n)
4

• where   0.2, and for sufficiently large n,

• a.f(n/b) = 3(n/4) lg(n/4)  (3/4) n lg n for c = 3/4
• case 3 applies:

• Thus the solution is T (n)  f (n)  when f (n)  n logb a  
• T(n) = (n lg n)