Unit VII

Undecidability and Intractability

Compiled By:
Ghanashyam BK
Computational Complexity
It is concerned with the study of the complexity of
computational tasks.
Complexity theoretic study looks at a task (or a class
of tasks) and at the computational resources required
to solve this task, rather than at a specific algorithm or
algorithmic scheme.
The complexity of computational problems can be
discussed by choosing a specific abstract machine as a
model of computation and considering how much
resource machine of that type require for the solution
of that problem.
Computational Complexity
Complexity Measure is a means of measuring the
resource used during a computation.
In case of Turing Machines, during any computation,
various resources will be used, such as space and time.
A problem is regarded as inherently difficult if solving
the problem requires a large amount of resources,
whatever the algorithm used for solving it.
In particular, computational complexity theory
determines the practical limits on what computers can
and cannot do.
Time complexity of A Turing Machine
The time complexity of a TM can be defined as;
Let T be a TM.
The time complexity of T is the function Tt defined on
the natural numbers as;
 for n ε N
 Tt(n) is the maximum number of moves T can make on any input
string of length n.
If there is an input string x such that for |x|=n,
T loops forever on input T, Tt(n) is undefined.
Space complexity of A Turing Machine
The space complexity of a TM can be defined as;
Let T be a TM.
The space complexity of T is the function St defined as
St (n) is the maximum number of the tape squares used
by T for any input string of length n.
If T is multi-tape TM, number of tape squares means
maximum of the number of individual tapes.
If for some input of length n, it causes T to loop forever,
St (n) is undefined.
Intractability
technique for solving problems not to be solvable in
polynomial time.
The problems that cab be solved by any computational
model, probably TM, using no more time then some
slowly growing function size of the input are called
“tractable”
i.e. those problems solvable within reasonable time
and space constraints (polynomial time).
The problems that cannot be solved in polynomial
time but requires super polynomial (exponential) time
algorithm are called intractable or hard problems.
 Problem and its types
Abstract Problem
Abstract problem A is binary relation on set of problem
instances, and the set S of problem solutions.
For e.g. Minimum spanning tree of a graph G can be viewed
as a pair of the given graph G and MST graph T.
Decision Problem
a problem that has an answer as either “true”, “yes”, “1” or
“false”, ”no”, “0”.
For e.g. if we have the abstract shortest path with instances
of the problem and the solution set as {0,1}, then we can
transform that abstract problem by reformulating the problem
as “Is there a path from u to v with at most k edges”.
In this situation the answer is either yes or no.
 Problem and its types
Optimization Problems
We encounter many problems where there are many feasible
solutions and our aim is to find the feasible solution with the best
value.
This kind of problem is called optimization problem
For e.g. given the graph G, and the vertices u and v find the
shortest path from u to v with minimum number of edges.
Function Problems
computational problem where a single output (of a total function)
is expected for every input, but the output is more complex than
that of a decision problem, that is, it isn't just YES or NO.
Notable examples include the Traveling salesman problem,
which asks for the route taken by the salesman
Complexity Classes
a complexity class is a set of problems of related
resource-based complexity. A typical complexity class
has a definition of the form:
“The set of problems that can be solved by an abstract
machine M using O(f(n)) of resource R, where n is the
size of the input.”
Types of Complexity Classes
 P Class
The P in the P class stands for Polynomial Time.
It is the collection of decision problems(problems with a “yes” or
“no” answer) that can be solved by a deterministic machine in
polynomial time.
 Features
The solution to P problems is easy to find.
P is often a class of computational problems that are solvable and
tractable. Tractable means that the problems can be solved in
theory as well as in practice. But the problems that can be solved
in theory but not in practice are known as intractable.
 Example: Calculating the greatest common divisor,
Finding a maximum matching.
Types of Complexity Classes
 NP Class
The NP in NP class stands for Non-deterministic Polynomial
Time
It is the collection of decision problems that can be solved by a
non-deterministic machine in polynomial time.
 Features:
The solutions of the NP class are hard to find since they are being
solved by a non-deterministic machine but the solutions are easy
to verify.
Problems of NP can be verified by a Turing machine in
polynomial time.
 Example: Boolean Satisfiability Problem (SAT),
Hamiltonian Path Problem, Graph coloring.
Types of Complexity Classes
NP-hard class
An NP-hard problem is at least as hard as the hardest
problem in NP and it is the class of the problems such that
every problem in NP reduces to NP-hard.
Features:
All NP-hard problems are not in NP.
It takes a long time to check them. This means if a solution
for an NP-hard problem is given then it takes a long time to
check whether it is right or not.
A problem A is in NP-hard if, for every problem L in NP,
there exists a polynomial-time reduction from L to A.
Example: Halting problem, No Hamiltonian cycle
Types of Complexity Classes
NP-complete class
A problem is NP-complete if it is both NP and NP-hard.
NP-complete problems are the hard problems in NP.
Features:
NP-complete problems are special as any problem in NP
class can be transformed or reduced into NP-complete
problems in polynomial time.
If one could solve an NP-complete problem in
polynomial time, then one could also solve any NP
problem in polynomial time.
Example: 0/1 Knapsack, Satisfiability, Vertex cover.
Types of Complexity Classes
Reducibility
way of converging one problem into another in such a way
that, a solution to the second problem can be used to solve
the first one.
A reduction is a transformation of one problem into
another problem.
For instance, if a problem X can be solved using an
algorithm for Y, X is no more difficult than Y, and we say
that X reduces to Y.
For example, the problem of squaring an integer can be
reduced to the problem of multiplying two integers. This
means an algorithm for multiplying two integers can be
used to square an integer.
Circuit Satisfiability
Boolean Satisfiability Problem:
Simply SAT is a problem of determining if a boolean
formula is satisfiable or unsatifiable
First problem that was proven to be NP-Complete.
Satisfiable
If the boolean variables can be assigned values such that
the formula turns out to be true, then we say that the
formula is satisfiable.
Unsatisfiable:
If it is not possible to assign such values, then we say
that the formula is unsatisfiable.
Circuit Satisfiability
Given a Boolean formula, determine whether this
formula is satisfiable or not.
N Boolean variables: x1, x2, x3, …. Xn
Literals: x1 or –x1
A clause: Disjunction of literals C1 v C2 v -C3
A formula: Conjunctive Normal form (CNF) C1^C2^C3
Examples:
F = x1 ^ -x1 is satisfiable if x1 = true and x2 is False
F= x1 ^ -x1 is unsatifyable
Circuit Satisfiability
Cook’s Theorem
Cook’s Theorem states that the Boolean satisfiability
problem is NP-complete.
i.e. it is in NP, and any problem in NP can be reduced in
polynomial time by a deterministic Turing machine to the
Boolean satisfiability problem.
Undecidability
a decision problem for which it is impossible to construct
a single algorithm that always leads to a correct “yes” or
“no” answer- the problem is not decidable.
An undecidable problem consists of a family of instances
for which a particular yes/no answer is required, such
that there is no computer program that, given any
problem instance as input, terminates and outputs the
required answer after a finite number of steps.
an undecidable problem is a problem whose language is
not a recursive set or computable or decidable.
the halting problem is undecidable for Turing machines.
Post’s Correspondence Problem
an undecidable decision problem that was Introduced by
Emil Post in 1946.
The input of the problem consists of two finite lists U=
{u1, u2, ….., un} and V= {v1, v2,……., vn} of words over
some alphabet Σ having at lest two symbols.
A solution to this problem is a sequence of indices i k;
1<=k<= n, for all k, such that
ui1 ui2 ………….. uik = vi1 vi2 ………….. Vik
We say i1, i2,…….ik is a solution to this instance of PCP.
Here, the decision problem is to decide whether such a
solution exits or not.
Post’s Correspondence Problem
Consider the following two lists (U,V):

U V
1 U1 a V1 baa
2 U2 ab V2 aa
3 U3 bba V3 bb

A solution to this problem would be the sequence (3,

2, 3, 1), because
u3u2u3u1 = bba + ab + bba + a = bbaabbbaa
v3v2v3v1 = bb + aa + bb + baa = bbaabbbaa
Post’s Correspondence Problem
Furthermore, since (3, 2, 3, 1) is a solution, so are all
of its "repetitions", such as (3, 2, 3, 1, 3, 2, 3, 1), etc.;
that is, when a solution exists, there are infinitely many
solutions of this repetitive kind.
However, if the two lists had consisted of only u2,u3
and v2,v3, then there would have been no solution
(because then no matching pair would have the same last
letter, as must occur at the end of a solution).
Halting Problem and its proof
“Given a program and an input to the program,
determine if the program will eventually stop when it is
given that input.”
The question is simply whether the given program will
ever halt on a particular input.
Trial Solution
Just run the program with the given input.
If the program stops we know the program halts.
But if the program does not stop in reasonable amount
of time, we can not conclude that it won’t stop. May be
we did not wait long enough!
Halting Problem and its proof
For example, in pseudocode, the program
while True: continue
does not halt; rather, it goes on forever in an infinite loop.
On the other hand, the program
print "Hello World!“
halts very quickly.
The halting problem is famous because it was one of the
first problems proven algorithmically undecidable.
This means there is no algorithm which can be applied
to any arbitrary program and input to decide whether the
program stops when run with that input
Halting Problem and its proof
This proof was devised by Alan Turing in 1936.
Suppose we have a solution to the halting problem
called H. H takes two inputs:
a program P and
an input I for the program P.
H generates an output "halt" if H determines that P
stops on input I or it outputs "loop" otherwise.
Halting Problem and its proof
So now H can be revised to take P as both inputs (the
program and its input) and H should be able to
determine if P will halt on P as its input.
Let, simple algorithm K that takes H's output as its
input and does the following
If H outputs "loop" then K halts,
Otherwise H's output of "halt" causes K to loop forever.
Halting Problem and its proof
That is, K will do the opposite of H's output.
function K(){
if (H()=="loop"){
return;
}
else {
while(true); //loop forever
}
}
Since K is a program, let us use K as the input to K.
Halting Problem and its proof
If H says that K halts then K itself would loop (that's
how we constructed it). If H says that K loops then K
will halt.
In either case H gives the wrong answer for K. Thus H
cannot work in all cases.
We've shown that it is possible to construct an input
that causes any solution H to fail.
Hence Proved