Undecidability &

Intractable Problems

Sunday, April 10, 2022 1

 Syllabus
A Language that is not recursively enumerable, An un-decidable
problem that is RE, Post Correspondence Problem, The Classes
P and NP : Problems Solvable in Polynomial Time, An Example:
Kruskal's Algorithm, Nondeterministic Polynomial Time, An NP
Example: The Traveling Salesman Problem, Polynomial-Time
Reductions NP Complete Problems, An NP-Complete Problem:
The Satisfiability Problem, Tractable and Intractable,
Representing Satisfiability, Instances, NP Completeness of the
SAT Problem, A Restricted Satisfiability Problem: Normal Forms
for Boolean Expressions, Converting Expressions to CNF, The
Problem of Independent Sets, The Node-Cover Problem.

Sunday, April 10, 2022 2

 Elements of the Chomsky Hierarchy

Sunday, April 10, 2022 3

 Decidable
A problem P is decidable if it can be solved by
a Turing machine T that always halt. (We say
that P has an effective algorithm.)

Note that the corresponding language of a

decidable problem is recursive.

Sunday, April 10, 2022 4

 Un-decidable
A problem is un-decidable if it cannot be
solved by any Turing machine that halts on all
inputs.

Note that the corresponding language of an

un-decidable problem is non-recursive.

Sunday, April 10, 2022 5

 Recursively Enumerable(RE) Language

A recursively enumerable language is a formal

language for which there exists a Turing machine
(or other computable function) that will halt and
accept when presented with any string in the
language as input but may either halt and reject or
loop forever when presented with a string not in
the language.
All regular, context-free, context-sensitive and
recursive languages are recursively enumerable.
Sunday, April 10, 2022 6
 Difference between Recursive & RE Language

• Recursively enumerable languages as one for which a

partial decider exists; that is, a Turing machine
which, given as input a word over your alphabet, will
either correctly accept/reject the word according to
your language, or if the word is not in your language,
it may loop forever.
• A recursive language, in contrast, is one for which a
total decider exists, i.e. one that will never loop, and
always halt in either an accepting or a rejecting state.

Sunday, April 10, 2022 7

 Complements of Recursive Languages

Theorem: If L is a recursive language, L is also

recursive.
Proof: Let M be a TM for L that always halt. We can
construct another TM M from M for L that always
halts as follows:

M
Accept Accept
Input M
Reject Reject

Sunday, April 10, 2022 8

 Complements of RE Languages
Theorem: If both a language L and its
complement L are RE, L is recursive.
Proof: Let M1 and M2 be TM for L and L respectively.
We can construct a TM M from M1 and M2 for L that
always halt as follows:

M
M1 Accept Accept
Input
M2 Accept Reject
Sunday, April 10, 2022 9
 Recursive

Recursively
Enumerable (RE)

Non-recursively
Enumerable (Non-RE)

Sunday, April 10, 2022 10

 A Non-recursive RE Language
• Recall that we can encode each TM uniquely
as a binary number and enumerate all TM’s as
T1, T2, …, Tk, … where the encoded value of the
kth TM, i.e., Tk, is k.
• Consider the language Lu:
Lu = {(k, w) | Tk accepts input w}
This is called the universal language.

Sunday, April 10, 2022 11

 Universal Language
• Note that designing a TM to recognize Lu is the
same as solving the problem of given k and w,
decide whether Tk accepts w as its input.
• Lu is RE but non-recursive, i.e., Lu can be
accepted by a TM, but there is no TM for Lu
that always halt.

Sunday, April 10, 2022 12

 Modified Post Correspondence
Problem
• We have seen an undecidable problem, that
is, given a Turing machine M and an input w,
determine whether M will accept w (universal
language problem).
• We will study another un-decidable problem
that is not related to Turing machine directly.

Sunday, April 10, 2022 13

Post Correspondence Problem (PCP)

Given two lists A and B:

A = w1, w2, …, wk B = x1, x2, …, xk
The problem is to determine if there is a
sequence of one or more integers i1, i2, …, im
such that:
w1wi1wi2…wim = x1xi1xi2…xim

(wi, xi) is called a corresponding pair.

Sunday, April 10, 2022 14
Example:
A B
i wi xi
1 100 001
2 11 111
3 111 11

PC-solution: A2 A1 A3  B2 B1 B3

11100111

Sunday, April 10, 2022 15

 Difference between PCP & MPCP
In the MPCP, a solution is required to start with
the first string on each list.

Sunday, April 10, 2022 16

 Example
A B
i wi xi
1 11 1
2 1 111
3 0111 10
4 10 0
This PCP instance has a solution: 3, 2, 2, 4:
w1w3w2w2w4 = x1x3x2x2x4 = 1101111110

Sunday, April 10, 2022 17

 Un-decidability of MPCP
To show that MPCP is un-decidable, we will
reduce the Universal Language Problem (ULP) to
MPCP:
Universal
Language A mapping MPCP
Problem (ULP)

If MPCP can be solved, ULP can also be solved.

Since we have already shown that ULP is un-decidable,
MPCP must also be un-decidable.
Sunday, April 10, 2022 18
 Problem?????

Def. :A Situation, Person or Thing that needs attention & needs to

be dealt with or solved?
Sunday, April 10, 2022 19
 Types of Complexity
• There are two types of Complexity
-Space Complexity
-Time Complexity

Sunday, April 10, 2022 20

 Algorithmic vs. Problem Complexity
• The Algorithmic complexity of a computation is
some measure of how difficult is to perform the
computation (i.e., specific to an algorithm)
• The complexity of a computational problem or
task is the complexity of the algorithm with the
lowest order of growth of complexity for solving
that problem or performing that task.
• Computational Complexity: deals with
classifying problems by how hard they are?.
Sunday, April 10, 2022 21
 Complexity definitions
Usually, the time required by an algorithm falls under three types −
• Best Case − Minimum time required for program execution.(Ω)
• Average Case − Average time required for program execution.(θ)
• Worst Case − Maximum time required for program execution.(Ο)

Sunday, April 10, 2022 22

Sunday, April 10, 2022 23
 Tractable vs. Intractable
• Polynomial Time Algorithm
-worst case complexity belongs to O(nk) for fixed integer k and
input size is n.

Tractable:
Set of all problems that can be solved in polynomial time are
called Tractable problem.
Intractable:
Set of all problems that can’t be solved in polynomial time are
called Intractable problem.
Sunday, April 10, 2022 24
 Optimization & Decision Problems
• Decision problems
– Given an input and a question regarding a problem,
determine if the answer is yes or no.
• Optimization problems
– Find a solution with the “best” value.
• Optimization problems can be cast as decision
problems that are easier to study
– E.g.: Shortest path: G = unweighted directed graph
• Find a path between u and v that uses the fewest edges
Sunday, April 10, 2022 25
 Class of “P” Problems
• Class P consists of (decision) problems that are
solvable in polynomial time
• Polynomial-time algorithms
– Worst-case running time is O(nk), for some constant k
(where n is the size of the input to the problem)
• Examples of polynomial time:
– O(n2), O(n3), O(1), O(n log n)
• Examples of non-polynomial time:
– O(2n), O(nn), O(n!)
Sunday, April 10, 2022 26
 Tractable/Intractable Problems
• Problems in P are also called tractable
• Problems not in P are intractable or unsolvable
– Can be solved in reasonable time only for small
inputs
– Or, can not be solved at all.

Sunday, April 10, 2022 27

 Example of Unsolvable Problem
• Turing discovered in the 1930’s that there are
problems unsolvable by any algorithm.The
most famous of them is the halting problem
– Given an arbitrary algorithm and its input, will
that algorithm eventually halt, or will it continue
forever in an “infinite loop”

Sunday, April 10, 2022 28

 Intractable Problems
• Can be classified in various categories based
on their degree of difficulty, e.g.,
– NP
– NP-complete
– NP-hard
• Let’s define NP algorithms and NP problems …

Sunday, April 10, 2022 29

 Nondeterministic and NP Algorithms
NP is the set of all decision problems solvable by a non-
deterministic algorithm in polynomial time.
Nondeterministic algorithm = two stage procedure:
1) Nondeterministic (“guessing”) stage:
generate randomly an arbitrary string that can be assumed of as a
candidate solution (“certificate”)
2) Deterministic (“verification”) stage:
take the certificate and the instance to the problem and returns YES if
the certificate represents a solution.
(Verification stage is polynomial)
•Sunday,
Warning:
April 10, 2022NP does not mean “non-polynomial” 30
 Relation between P & NP
• NP is the set of all decision problems solvable
by nondeterministic algorithms in polynomial
time.
• Deterministic algorithms is just a special case
of non deterministic one..

Sunday, April 10, 2022 31

 P=NP????
• The P versus NP problem is a major unsolved
problem in computer science.
• Computer scientists do not know for certain
whether P = NP or not.

Sunday, April 10, 2022 32

Sunday, April 10, 2022 33
 Example of P Class
Kruskal’s algorithm

1. Select the shortest edge in a network.

2. Select the next shortest edge which does not create

a cycle.

3. Repeat step 2 until all vertices have been connected.

Complexity is O(elog e) where e is the number of edges

Sunday, April 10, 2022 34
Example
A cable company want to connect five villages to their network which
currently extends to the market town of Avonford. What is the minimum
length of cable needed?

Brinleigh 5
Cornwell

3
4
8 6

8
Avonford Fingley Donster
7

5
4
2

Sunday, April 10, 2022

Edan 35
 We model the situation as a network, then the problem is to find
the minimum connector for the network

B 5 C

3
4
8 6

8
A F D
7
5
4 2

Sunday, April 10, 2022

E 36
 Kruskal’s Algorithm
List the edges in
B 5 order of size:
C

3 ED 2
4
8 6 AB 3
8 AE 4
A
7 F D CD 4
BC 5
5 EF 5
4
2 CF 6
AF 7
E BF 8
Sunday, April 10, 2022 CF 8 37
 Kruskal’s Algorithm
Select the shortest
B 5 edge in the
C
network
3
4
8 6 ED 2
8
A F D
7
5
4
2

E
Sunday, April 10, 2022 38
 Kruskal’s Algorithm
Select the next
B 5 shortest edge
C
which does not
3 create a cycle
6 4
8
8 ED 2
A
7 F D AB 3

5
4
2

E
Sunday, April 10, 2022 39
 Kruskal’s Algorithm
Select the next
B 5 shortest edge
C
which does not
3 create a cycle
6 4
8
8 ED 2
A
7 F D AB 3
CD 4 (or AE 4)
5
4
2

E
Sunday, April 10, 2022 40
 Kruskal’s Algorithm
Select the next
B 5 shortest edge
C
which does not
3 create a cycle
6 4
8
8 ED 2
A
7 F D AB 3
CD 4
5 AE 4
4
2

E
Sunday, April 10, 2022 41
 Kruskal’s Algorithm
Select the next
B 5 shortest edge
C
which does not
3 create a cycle
6 4
8
8 ED 2
A
7 F D AB 3
CD 4
5 AE 4
4
2 BC 5 – forms a
cycle
E EF 5
Sunday, April 10, 2022 42
 Kruskal’s Algorithm
All vertices have
B 5 been
C
connected.
3
4
8 6 The solution is
8
A
7 F D ED 2
AB 3
5 CD 4
4
2 AE 4
EF 5
E Total weight of
Sunday, April 10, 2022 tree: 18 43
 Example of NP Class
The Travelling Salesman Problem describes a salesman
who must travel between N cities. The order in which
he does so is something he does not care about, as long
as he visits each one during his trip, and finishes where
he was at first.

Sunday, April 10, 2022 44

 E.g.: Hamiltonian Cycle
• Given: a directed graph G = (V, E), determine a
simple cycle that contains each vertex in V
– Each vertex can only be visited once
• Certificate: hamiltonian

– Sequence: v1, v2, v3, …, v|V|

not
hamiltonian

Sunday, April 10, 2022 45

 NP-Completeness (informally)
P NP-complete
• NP-complete problems are
NP
defined as the hardest
problems in NP
• Most practical problems turn out to be either P
or NP-complete.
• Study NP-complete problems …
Sunday, April 10, 2022 46
 Reductions
• Reduction is a way of saying that one problem is
“easier” than another.
• We say that problem A is easier than problem B,
(i.e., we write “A  B”)
if we can solve A using the algorithm that solves
B.
• Idea: transform the inputs of A to inputs of B
yes
yes
 
f Problem B no
no
Sunday, April 10, 2022 Problem A 47
 Polynomial Reductions

• Given two problems A, B, we say that A is

polynomially reducible to B (A p B) if:

1. There exists a function f that converts the input of A

to inputs of B in polynomial time

2. A(i) = YES  B(f(i)) = YES

Sunday, April 10, 2022 48
 NP Complete & Hard

• A problem B is NP-complete if:

(1) B  NP

(2) A p B for all A  NP

• If B satisfies only property (2) we say that B is

NP-hard

Sunday, April 10, 2022 49

 Relation between P,NP,NP-
Complete, NP-Hard

Sunday, April 10, 2022 50

 NP-Complete & NP-Hard Problems
• NP-Complete Problems
Following are some NP-Complete problems.
I. Determining whether a graph has a Hamiltonian cycle.
II. Determining whether a Boolean formula is satisfiable etc.

• NP-Hard Problems
The following problems are NP-Hard
I. The circuit-satisfiability problem
II. Set Cover
III.Vertex Cover

Sunday, April 10, 2022 51

 TSP is NP-Complete
First we have to prove that TSP belongs to NP. In
TSP, we find a tour and check that the tour contains
each vertex once. Then the total cost of the edges of
the tour is calculated. Finally, we check if the cost is
minimum. This can be completed in polynomial time.
Thus TSP belongs to NP.
TSP reduce into Hamiltonian Cycle where simple
cycle that contains each vertex can only be visited
once.
Sunday, April 10, 2022 52
 Satisfiability Problem (SAT)
• Satisfiability problem: given a logical expression ,
find an assignment of values (F, T) to variables x i
that causes  to evaluate to T
 =x1   x2  x3   x4

• SAT was the first problem shown to be NP-complete!

• Video

Sunday, April 10, 2022 53

 Normal Forms for Boolean
Expression
Essential definitions
A literal: is either a variable, or a negated
e.g. ¬x, x
 A clause: is the logical OR of one or more literals.
e.g. x , x ˅ ¬y ˅ z.
 A boolean expression: is said to be in conjunctive
normal form or CNF, if it is the AND of clauses.

Sunday, April 10, 2022 54

 Important notation

OR () is treated as a Sum, using the (+) operator.

AND () is treated as a product.
normally use juxtaposition (no operator).
Not = (¬)

Sunday, April 10, 2022 55

 Example
The expression (x ˅ ¬y) ˄ (¬x ˅ z):
will be written in CNF as (x + ¬y)(¬x + z)
It is in Conjunctive Normal Form(CNF), since it is
the AND (product) of the clauses:
(x + ¬y) and (¬x + z).

Sunday, April 10, 2022 56

 Question?
are the expressions
(x + y¬z)(x + y + z)(¬y + ¬z), xyz in CNF?
The first expression not in CNF.
It is the AND of three subexpressions, the last two are
clauses, but the first is not, it is the sum of a literal and a
product of two literals.
 The second expression in CNF.
the clause can have only one literal, so it is the product
of three clauses, (x ), (y), and (z)

Sunday, April 10, 2022 57

 CNF Satisfiability
• CNF is a special case of SAT.
•  is in “Conjunctive Normal Form” (CNF)
– “AND” of expressions (i.e., clauses)
– Each clause contains only “OR”s of the variables
and their complements

E.g.:  = (x1  x2)  (x1   x2)  ( x1   x2)

clauses
Sunday, April 10, 2022 58
 DeMorgan’s Laws
I. ¬(E  F) ⇒ ¬(E)  ¬(F)
II. ¬(E  F) ⇒ ¬(E)  ¬(F)
III. ¬(¬E) ⇒ E : the double negation.

Sunday, April 10, 2022 59

 Example
E = ¬((¬(x + y))(¬x + y))
¬((¬(x + y))(¬x + ¬y)) Start

¬(¬(x + y))+¬(¬x + ¬y) (1)

x + y + ¬(¬x + ¬y) (3)

x + y + (¬(¬x))(¬(¬y)) (2)

x + y + x +y (3)
Sunday, April 10, 2022 60
 IndepIndependent Setnt Set
An independent set is a set of nodes with no edges
between them

This graph
contains an
independent
set of size 3

Sunday, April 10, 2022 61

 3-CNF Satisfiability
A subcase of CNF problem:
– Contains three clauses
• E.g.:
 = (x1  x1  x2)  (x3  x2  x4) 
(x1  x3   x4)

• 3-CNF is NP-Complete

Sunday, April 10, 2022 62

 Vertex/Node Cover Problem
• A vertex cover of an undirected graph is a
subset of its vertices such that for every edge
(u, v) of the graph, either ‘u’ or ‘v’ is in vertex
cover. Although the name is Vertex Cover, the
set covers all edges of the given graph.

Sunday, April 10, 2022 63

 Example:

Sunday, April 10, 2022 64

Sunday, April 10, 2022 65