Modified by Dr.

ISSAM ALHADID
15/4/2019
 Decision and Optimization Problems.
 Polynomial Time vs. Non-Polynomial Time.
 Three General Problem Categories.
 The Sets P and NP.
 NP-Complete Problems.
 NP-Hard Problems.
 NP-Hard vs. NP-Complete Problems.
 Most of the problems we have dealt with thus
far in the class are optimization problems, in
which the goal is to find the maximum or
minimum value of a quantity and maybe also
find the particular solution that achieves this
maximum/minimum value. For example, in
the SHORTESTPATH problem, we are given an
undirected graph G = (V, E) and vertices s and
t, and we wish either to find the length of the
shortest path from s to t, or to find the
shortest path from s to t itself>
 decision problems, in which the answer is
simply “yes” or “no” (or formally 1 or 0). This
is not restrictive at all: we can always cast a
given optimization problem as a related
decision problem by imposing a bound on
the value to be optimized.
 For example, a decision problem related to
SHORTEST-PATH is PATH: given a directed
graph G = 1 (V, E), vertices s and t, and an
integer k, does a path exist from s to t that
consists of at most k edges?
 There is an interesting relationship between a
decision problem and its related optimization
problem: the decision problem is in a sense “easier”,
or at least “no harder”.
 As a specific example, we can solve PATH by solving
SHORTEST-PATH and then comparing the number of
the edges in the shortest path found to the value of
the decision-problem parameter k.
 Stated in a different way, if an optimization problem
is easy, its corresponding decision problem is also
easy.
 A decision problem has only two possible
solutions: “Yes” or “No”.
 Example: TSP

 Questions: Is there a “tour” of all cities in C

having total length no more than B?
 Optimization problems, which means that the
output is an optimal solution.
 Decision problems:
◦ Answer is yes or no.
 Optimization problems:
◦ Answer is a number representing an
objective value (Max. or Min)
 Each optimization problem has a corresponding
decision problem.
◦ That is a decision problem can be derived from the
corresponding optimization problem.
 Thus, a solution to an optimization problem
produces a solution to the corresponding
decision problem.
 (Back to Slide 5)
 Example: TSP
 If we learned that the total weight of an
optimal tour for a particular instance of TSP
Optimization problem was 120, the answer to
the corresponding decision problem would be
“yes” for 120 ≤ d and “no” otherwise. (the
tour is not greater than d)
 Or decision problem would be “yes” for d ≤
120 and “no” otherwise. (the tour is not less
than d)
 Example: 0/1 Knapsack
 If we learned that the optimal profit for an
instance of the 0/1 Knapsack Optimization
problem was $230, the answer to the
corresponding decision problem would be
“yes” for P ≤ $230 and “no” otherwise.
 (total profit at least equal to P).
 Suppose we have chosen some problem, and
for this problem we have defined an algorithm
to solve it.
 We want to have a (rough) estimate of the
number of elementary computations (additions,
multiplications, comparisons, etc.) that are
required by the algorithm to solve the problem
in the worst case .
 This expression will depend on the values of
the parameters input size.
 Polynomial functions: Any function that is O(n k ),
i.e. bounded from above by n k for some constant
k. E.g. O(1), O(log n), O(n), O(n × log n), O(n2),
O(n3).
 we defined ‘polynomial’ to be any function of the
form :
 Exponential functions: The remaining functions.
E.g. O(2n), O(n!), O(nn)
Why have we grouped functions together into these two broad classes?
The next two tables and the graph attempt to show you why.
 On the basis of this classification of functions into
polynomial and exponential, we can classify
algorithms:
1. Polynomial-Time Algorithm: an algorithm whose
order-of-magnitude time performance is
bounded from above by a polynomial function of
n, where n is the size of its inputs
2. Exponential Algorithm: an algorithm whose
order-of-magnitude time performance is not
bounded from above by a polynomial function of
n.
 Why do we divide algorithms into these two broad
classes? The next table, which assumes that one
instruction can be executed every microsecond,
attempt to show you why.
 And, in a similar way, we can classify problems into
two broad classes:
1. Tractable Problem: a problem that is solvable by a
polynomial-time algorithm. The upper bound is
polynomial.
2. Intractable Problem: a problem that cannot be
solved by a polynomial-time algorithm. The lower
bound is exponential. (problems for which there
exist no efficient algorithms to solve them)
 Examples of tractable problems: Searching an
unordered list, Searching an ordered list, Sorting a list
 Examples of intractable problems: List all permutations
_____(all possible orderings) of n numbers.
 Most intractable problems have an algorithm –
the same algorithm – that provides a solution, and
that algorithm is the brute-force search.
 This algorithm, however, does not provide
an efficient solution and is, therefore, not feasible
for computation with anything more than the
smallest input.
 The problems that cannot be solved in polynomial
time are termed as intractable problems.
 For example, performing a brute-force approach on
the Rivest Shamir Adleman (RSA ) algorithm to
decrypt the encrypted content. This indicated that
this can be achieved but the time taken for that
brute-force is not determined. An NP-complete
problems is also considered as intractable problems.
 However, we can create extreme examples in which a
non-polynomial- time algorithm is better than a
polynomial-time algorithm for practical input sizes:
◦ 2(n) > n10 for some large value !
 When we are determining whether an algorithm is
polynomial time, it is necessary to be careful about the
input size.
 There are three general categories of problems as
far as intractability is concerned:
1. Problems for which polynomial-time
algorithms have been found.
2. Problems that have been proven to be
intractable.
3. Problems that have not been proven to be
intractable, but for which polynomial-time
algorithms have never been found.

 Most problems in computer science fall into either

first or third category.
 Intractability problems categories examples:
◦ 1- Subset Sum
◦ The input is a positive integer C and n objects whose
sizes are positive integers s1, s2,..., sn. Optimization
Problem: Among all subsets of objects with sum at
most C, what is the largest subset sum?
◦ Decision Problem: Is there a subset of objects whose
sizes add up to exactly C?

 Examples: http://lcm.csa.iisc.ernet.in/dsa/node219.html
 1st: Problems for which polynomial-time
algorithms have been found:
 2nd: Problems that have been proven to be
intractable:
◦ There are two types of problems of this category:
 The first type is problems that require a non-
polynomial amount of output.
 For example, the problem of determining all
Hamiltonian cycles. If there was an edge from every
vertex to every other vertex, there would be (n – 1)!
such cycles.
 To solve the problem, an algorithm would have to
output all of these cycles.
 Example: for 51 vertices… we will have …. Cycles !
 2nd: Problems that have been proven to be
intractable:
◦ There are two types of problems of this category:
 The second type occurs when we are not asking for a
non-polynomial amount of output, and we can prove
that the problem cannot be solved in polynomial
time.
 3rd: Problems that have not been proven to be
intractable, but for which polynomial-time
algorithms have never been found:
◦ For example, if we state the problems to require one
solution, the 0/1 knapsack problem, the TSP
problem, and the Hamiltonian cycles problem all fall
into this category.
 Definition: P is the set of all decision problems that
can be solved by Polynomial-time algorithms.
 However, could some decision problem for which we
have not found a polynomial-time algorithm also be in
P?
 For example, could the TSP Decision problem be in P?
 An algorithm composed of the following two
separate stages:
1. Guessing (Non-Deterministic) Stage: Given
an instance of a problem, this stage simply
produces some string S. The string can be
thought of as a guess at a solution to the
instance. However, it could just be a string
of nonsense.
 An algorithm composed of the following two
separate stages:
2. Verification (Deterministic) Stage: The
instance and the string S are the input to
this stage. This stage then proceeds in an
ordinary deterministic manner either:
 A non-deterministic algorithm “solves” a
decision problem if:
◦ For any instance for which the answer is “yes” there
is some string S for which the verification stage
returns “true”.
◦ For any instance for which the answer is “no” there
is no string for which the verification stage returns
“true”.
 Definition: A polynomial-time non- deterministic
algorithm is a non-deterministic algorithm whose
verification stage is a polynomial-time algorithm.

 Definition: NP is the set of all decision problems

that can be solved by polynomial-time non-
deterministic algorithms.

 Note that NP stands for “Non-deterministic

Polynomial”.
 The TSP Decision problem is in NP.
◦ This does not mean that we have a polynomial-time
◦ algorithm that solves the problem.
 There are thousands of other problems that
no one has been able to solve with
polynomial-time algorithms but that have
been proven to be in NP because polynomial-
time non-deterministic algorithms have been
developed for them.
 That is every problem in P is also in NP.
◦ This trivially (insignificantly) true because any
problem in P can be solved by a polynomial-time
algorithm.
 Figure 1: shows the set of all decision
problems.
 That is no one has ever proven that there is a
problem in NP that is not in P.
 Therefore, NP – P may be empty.
 The question of whether P = NP?
 This question is important, because most
decision problems are in NP.
◦ Therefore, if P = NP we would have polynomial-
time
◦ algorithms for most known decision problems.
 To show that P ≠ NP we would have to find a
problem in NP that is not in P, whereas to
show that P = NP we would have to find a
polynomial-time algorithm for each problem
in NP.
 Definition: If there exists a polynomial-time
transformation algorithm from decision
problem A to decision problem B, problem A
is polynomial-time many-one reducible to
problem B:
◦ That is A α B, means problem A reduces to problem
B.
◦ “Many-one” means a transformation algorithm is a
function that may map many instances of problem
A to one instance of problem B. That is, it is a
many-one function.
 Example: Hamiltonian cycle and TSP.
 Hamiltonian cycle is one that visits every
node of graph exactly once, except first node
is same as last one.
 A graph does not have to be complete for it
to have a Hamiltonian cycle.
 We can reduce the Hamiltonian cycle problem
to TSP problem in the following way:
 The power of the reduction process is that if any
NP-Complete problem can be reduced to a class
P problem, then all of the NP problems must
have polynomial-time solutions.
 So far, all attempts at reductions of this type
have failed
 Definition: A problem B is called NP-Complete if
both of the following are true:
1) B is in NP.
2) For every other problem A in NP, A α B.
(means problem A reduces to problem B)
(Brute Force)
 P is a complexity class that represents the set
of all decision problems that can be solved in
polynomial time.

 NP is a complexity class that represents the

set of all decision problems for which the
instances where the answer is "yes" have
proofs that can be verified in polynomial
time.
 NP-Complete is a complexity class which
represents the set of all problems X in NP for
which it is possible to reduce any other NP
problem Y to X in polynomial time.

 this means that we can solve Y quickly if we

know how to solve X quickly.
 NP-hard these are the problems that are at
least as hard as the NP-complete problems.
Note that NP-hard problems do not have to
be in NP, and they do not have to be decision
problems
 The precise definition here is that a
problem X is NP-hard, if there is an NP-
complete problem Y, such that Y is reducible
to X in polynomial time.
 NP-hard
 But since any NP-complete problem can be
reduced to any other NP-complete problem in
polynomial time, all NP-complete problems
can be reduced to any NP-hard problem in
polynomial time. Then, if there is a solution
to one NP-hard problem in polynomial time,
there is a solution to all NP problems in
polynomial time.