Analysis of Algorithms

(CSE 2007)

Module 5
Limitations of
Algorithm

1
 Module 5-Content
1. Introduction, Lower Bound Arguments
2. Decision Trees
P, NP ,NP complete problems
3. Backtracking- Introduction
N Queens Problem
N Queens Algorithm
4. Travelling Salesman Problem

2
Lower Bounds
Lower bound: an estimate on a minimum amount of work
needed to solve a given problem

Examples:
• number of comparisons needed to find the largest element in
a set of n numbers
• number of comparisons needed to sort an array of size n
• number of comparisons necessary for searching in a sorted
array
• number of multiplications needed to multiply two n-by-n
matrices

3
Lower Bounds (cont.)
Lower bound can be
• an exact count
• an efficiency class ()
Tight lower bound: there exists an algorithm with the
same efficiency as the lower bound

Problem Lower bound Tightness

sorting (comparison-based) (nlog n) yes
searching in a sorted array (log n) yes
element uniqueness (nlog n) yes
n-digit integer multiplication (n) unknown
multiplication of n-by-n matrices (n2) unknown

4
Decision Trees
Decision tree — a convenient model of algorithms
involving comparisons in which:
• internal nodes represent comparisons
• leaves represent outcomes (or input cases)
Decision tree for 3-element insertion sort
abc
yes no
a<b

abc bac
yes no yes no
b< c a<c

a<b<c acb bca

b<a<c
yes no yes no
a<c b<c

a<c<b c<a<b b<c<a c<b<a

5
Decision Trees and Sorting Algorithms
• Any comparison-based sorting algorithm can be
represented by a decision tree (for each fixed n)
• Number of leaves (outcomes)  n!
• Height of binary tree with n! leaves  log2n!
• Minimum number of comparisons in the worst case 
log2n! for any comparison-based sorting algorithm,
since the longest path represents the worst case and its
length is the height

6
Types of Problems
• Deterministic Problem
• Non Deterministic Problem
• Polynomial Problem
• Exponential Problem
 Deterministic problem
• In deterministic algorithm, for a given particular
input, algorithm will always produce the same
output going through same set of states.
Particular input -- computer -- always produce the
same output
Can determine next step of solution.

Ex: sum of n numbers

Non-Deterministic Problem
• In non- deterministic for same input, compiler
may produce different output.
Input ---- Computer ---- different output
Cannot determine next step of solution

Ex: Any program using Random Function.

Polynomial Problem
• Problem is said to be in polynomial, if there
exists an algorithm that solves the problem in
time
T(n) = O (n^k)
Where k is constant
Ex: Quicksort O(n^2), Searching O(n)
Exponential Problem
• Problem is said to be exponential, if no
polynomial time algorithm can be developed
for it or algorithm that exists can solve the
problem in
T(n) = O(2^n) or O(k^n)
Ex: Knapsack or Graph Coloring
 Class P

P: is a set of problems that can be solved in polynomial

time using Deterministic algorithms , also P class
problems can be verified in polynomial time.
Examples:
searching
element uniqueness
graph connectivity
graph acyclicity
 Class NP
NP (nondeterministic polynomial): is a set of problems that
can be solved using non deterministic algorithms in
polynomial time. Solution is difficult to find with long time
(exponential) but answer can be verified in polynomial time.
Hard to solve, but easy to verify.
A nondeterministic polynomial algorithm is an abstract two-
stage procedure that:
• generates a solution of the problem (on some input) by
guessing
• checks whether this solution is correct in polynomial time
By definition, it solves the problem if it’s capable of generating
and verifying a solution on one of its tries.
What other problems are in NP?
• Hamiltonian circuit existence
• Partition problem: Is it possible to partition a set of
n integers into two disjoint subsets with the same
sum?
• Decision versions of TSP, knapsack problem, graph
coloring, and many other combinatorial
optimization problems.
• All the problems in P can also be solved in this
manner (but no guessing is necessary), so we have:
P  NP
• Big (million dollar) question: P = NP ?
NP-Complete Problems
A decision problem D is NP-complete if it is as hard as any
problem in NP, i.e.,
• D is in NP
• every problem in NP is polynomial-time reducible to D
NP problems

NP-complete
problem

15
 NP-Complete Problems (cont.)
Other NP-complete problems obtained through
polynomial-time reductions from a known NP-complete
problem
NP problems

know n
NP-complete
problem
candidate
f or NP -
completeness

Examples: TSP, knapsack, partition, graph-coloring and

hundreds of other problems of combinatorial
nature

16
NP Hard Problem
• A problem H is NP hard , if every problem L in NP
can be reducible to H in polynomial time. H’s
solution can be used to solve L in polynomial
time.

Ex: sum of subset problem

17
 Backtracking
• A given problem has a set of constraints and possibly an
objective function
• The solution optimizes an objective function, and/or is
feasible.
• We can represent the solution space for the problem
using a state space tree
• The root of the tree represents 0 choices,
• Nodes at depth 1 represent first choice
• Nodes at depth 2 represent the second choice, etc.
• In this tree a path from a root to a leaf represents a
candidate solution
• An example for Backtracking is N Queens
problems.

18
N Queens Problem
1. The n-queens problem defines to place n queens
on an n×n chessboard so that no two queens
attack each other by being in the same row or in
the same column or on the same diagonal.
2. For n=1, the problem has a trivial solution, and
it is easy to see that there is no solution for n=2
and n=3.
3. So let us consider the four-queens problem and
solve it by the backtracking technique.

19
• We start with the empty board and then place queen 1 in the
first possible position of its row, which is in column 1 of row
1.
• Then we place queen 2, after trying unsuccessfully columns1
and 2,in the first acceptable position for it,which is square (2,
3), the square in row 2 and column 3.
• This proves to be a dead end because there is no acceptable
position for queen 3. So, the algorithm backtracks and puts
queen 2 in the next possible position at (2, 4).
• Then queen 3 is placed at (3, 2), which proves to be another
dead end.
• The algorithm then backtracks all the way to queen 1 and
moves it to (1, 2).
• Queen 2 then goes to (2, 4), queen 3 to (3, 1), and queen 4 to
(4, 3), which is a solution to the problem

20
Queens Algorithm
AlGORITHM Backtrack(X[l..i])
//Gives a template of a generic backtracking algorithm
//Input: X[Li] specifies first i promising components of a solution
//Output: All the tuples representing the problem's solutions
if X[Li] is a solution write X[l..i]
else
for each element x E S;+J consistent with X[Li] and the constraints do
X[i + 1] *-X
Backtrack(X[1..i + 1 ]J

21
4*4 Queen Problem using State space tress

22
• For TSP pls refer textbook

23
Thank You