Unit 5 - Theory of Computation FINAL
Unit 5 - Theory of Computation FINAL
Subject Notes
CS-501- Theory of Computation
B. Tech V semester
Turing Machine: Techniques for construction. Universal Turing machine Multi-tape, multi-head
and multidimensional Turing machine, N-P complete problems. Decidability and Recursively
Enumerable Languages, decidability, decidable languages, undecidable languages, Halting problem
of Turing machine & the post correspondence problem.
………………………………………………………………………………………………
Objective: To develop an overview of how automata theory, languages and computation are
applicable in engineering application.
………………………………………………………………………………………………
Unit-V: Turing Machine
Introduction:
Turing machine is considered as a simple model of a real computer. Turing machines can be used
to accept all context-free languages, but also languages such as L = {ambncmn: m ≥ 0, n ≥ 0} which
is not class of language comes under regular and context free. Every problem that can be solved on
a real computer can also be solved by a Turing machine.
B B a a a b b a B B
Tape Head
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering Page 1
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering Page 2
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 3
CS-501 Theory of Computation AIST SAGAR
q3 - (q0, b, (q5, b,
R) R)
q4 (q5 ,0, (q4 ,0, (q4 ,1,
R) R) R)
q5 (q2 ,0, - -
L)
Table 5.1: Computation
string w = 00
q100 |- 0q10 |- 00q1 |- 0q201 |- q2001 |- q2b001 |- q3001 |- q401 |- 0q41 |- 01q4 |- 010q5 |-
01q200 |-*- q5000
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 4
CS-501 Theory of Computation AIST SAGAR
Figure 5.2:
Transition Diagram Acceptance of Language by
Turing Machine:
A TM accepts a language if it enters into a final state for any input string w. A language is
recursively enumerable(generated by Type-0 grammar) if it is accepted by a Turing machine.
A TM decides a language if it accepts it and enters into a rejecting state for any input not in the
language. A language is recursive if it is decided by a Turing machine. There may be some
cases where a TM does not stop. Such TM accepts the language, but it does not decide it.
Example 1:
Design a TM to recognize all strings consisting of an
odd number of α’s. Solution:
• The Turing machine M can be constructed by the following moves:
• Let q1 be the initial state.
• If M is in q1; on scanning α, it enters the state q2 and writes B (blank).
• If M is in q2; on scanning α, it enters the state q1 and writes B (blank).
From the above moves, we can see that M enters the state q1 if it scans an even number of
α’s, and it enters the state q2 if it scans an odd number of α’s. Hence q2 is the only accepting
state.
Hence,
M = {{q1, q2}, {1}, {1, B}, δ, q1, B, {q2}} where δ is given by:
Example 2:
Design a Turing Machine that reads a string representing a binary number and erases all leading
0’s in the string. However, if the string comprises of only 0’s, it keeps one 0.
Solution:
Let us assume that the input string is terminated by a blank symbol, B, at each end
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 5
CS-501 Theory of Computation AIST SAGAR
of the string. The Turing Machine, M, can be constructed by the following moves:
• Let q0 be the initial state.
• If M is in q0, on reading 0, it moves right, enters the state q1 and erases 0. On reading
1, it enters the state q2 and moves right.
• If M is in q1, on reading 0, it moves right and erases 0, i.e., it replaces 0’s by B’s. On
reaching the leftmost 1, it enters q2 and moves right. If it reaches B, i.e., the string
comprises of only 0’s, it moves left and enters the state q3.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 6
CS-501 Theory of Computation AIST SAGAR
Tape Present State Present State Present State Present State Present State
Alphabet “q0” “q1” “q2” “q3” “q4”
Symbol
0 BRq1 BRq1 0Rq2 - 0Lq4
1 1Rq2 1Rq2 1Rq2 - 1Lq4
B BRq1 BLq3 BLq4 0Lqf BRqf
Table 5.3: Example of Turing Machine
Techniques of Construction: Multi-tape Turing Machine:
Multi-tape Turing Machines have multiple tapes where each tape is accessed with a separate
head. Each head can move independently of the other heads. Initially the input is on tape 1 and
others are blank. At first, the firsttape is occupied by the input and the other tapes are kept blank.
Next, the machine reads consecutive symbols under its heads and the TM prints a symbol on
each tape and moves its heads.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 7
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 8
CS-501 Theory of Computation AIST SAGAR
Note: Every Multi-tape Turing machine has an equivalent single-tape Turing machine.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 9
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 10
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 11
CS-501 Theory of Computation AIST SAGAR
Figure 5.5: Offline Turing Machine Equivalence of Single Tape & Multi Tape Turing
Machine
In the classical framework k-tape Turing machines have the same computational power of
Single-tape Turing machines and given a Multi-tape Turing Machine MK it is always possible
to define a Single-tape Turing Machine which is able to fully simulate its behavior and therefore
to completely execute its computations. The Gross-one
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 12
CS-501 Theory of Computation AIST SAGAR
methodology allows us to give a more accurate definition of the equivalence among different
machines as it provides the possibility not only to separate different classes of infinite sets with
respect to their cardinalities but also to measure the number of elements of some of them. With
reference to Multi-tape Turing machines, the Single-tape Turing Machines adopted for their
simulation use a particular kind of tape which is divided into tracks (multitrack tape). In this
way, if the tape has m tracks, the head is able to access (for reading and/or writing) all the m
characters on the tracks during a single operation. This tape organization leads to a
straightforward definition of the behavior of a Single-tape Turing machine able to completely
execute the computations of a given Multi-tape Turing machine
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 13
CS-501 Theory of Computation AIST SAGAR
Decidable Language
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 14
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 15
CS-501 Theory of Computation AIST SAGAR
Undecidable Language:
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 16
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 17
CS-501 Theory of Computation AIST SAGAR
P & NP Problems:
If a problem can be solved in polynomial time, it is said to belong to the P class of problems
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 18
CS-501 Theory of Computation AIST SAGAR
P-type problems
are tractable.
For some intractable problems, you can verify that the solution is correct using a P-type
algorithm. For example, you can verify that a given solution to the TSP visits every city. These
problems are referred to as Non- deterministic Polynomial problems or NP-type problems. The
challenge for programmers is to find a P-type solution to NP-type problems.
P Problems:
As the name says these problems can be solved in polynomial time, i.e.; O(n), O(n2) or O(nk)
where k is a constant.
NP Problems:
Some people think NP as Non-Polynomial. But actually, it is Non-deterministic Polynomial
time. i.e.; “yes” instances of these problems can be solved in polynomial time by a non-
deterministic Turing machine and hence can take up to exponential time (some problems can be
solved in sub-exponential but super polynomial time) by a deterministic Turing machine. In
other words, these problems can be verified (if a solution is given, say if it is correct or wrong)
in polynomial time. Examples include all P problems. One example of a problem not in P but in
NP is Integer Factorization.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 19
CS-501 Theory of Computation AIST SAGAR
From the figure 5.10, it’s clear that NPC problems are the hardest problems in NP while being
the simplest ones in NPH. i.e.; NP∩NPH=NPC
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 20
CS-501 Theory of Computation AIST SAGAR
Given a general problem, we can say it’s in NPC, if and only if we can reduce it to some NP
problem (which shows it is in NP) and also some NPC problem can be reduced to it (which shows
all NP problems can be reduced to this problem).
Also, if a NPH problem is in NP, then it is NPC
Examples of NP Problems:
Boolean Satisfiability Problem:
Boolean Satisfiability or simply SAT is the problem of determining if a Boolean formula is
satisfiable or unsatisfiable.
Satisfiable: If the Boolean variables can be assigned values such that the formula turns out to be
TRUE, then we say that the formula is satisfiable.
Unsatisfiable: If it is not possible to assign such values, then we say that the formula is
unsatisfiable.
2- SAT Problem:
• 2-SAT is a special case of Boolean Satisfiability Problem and can be solved in
polynomial time.
• To understand this better, first let us see what is Conjunctive Normal Form (CNF) or also
known as Product of Sums (POS).
CNF: CNF is a conjunction (AND) of clauses, where every clause
is a disjunction (OR). F = (A1 V B1) ^ (A2 V B2) ^ (A3 V B3)
.........................................................................(Am V Bm)
• Now, 2-SAT limits the problem of SAT to only those Boolean formula which are
expressed as a CNF with every clause having only 2 terms (also called 2-CNF)
• For 2-SAT problem the CNF value is TRUE, if value of every clause is TRUE. Let one
of the clauses be (A V B) so we can say (A V B) = TRUE in following two conditions
▪ If A = 0, B must be 1 i.e. (A’ => B)
▪ If B = 0, A must be 1 i.e. (B’ => A)
Thus (A V B) is true equivalent to (A’ => B) ^ (B’ => A)
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 21
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 22
CS-501 Theory of Computation AIST SAGAR
Vertex Cover Problem is a known NP Complete problem, i.e., there is no polynomial time
solution for this unless P = NP. Although There can be an approximate polynomial time
algorithm to solve the problem. Following is a simple approximate algorithm.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 23
CS-501 Theory of Computation AIST SAGAR
time.
Therefore, Hamiltonian Cycle ∈ NP.
Prove Hamiltonian Cycle
Problem ∈ NP-Complete
Reduction: Vertex Cover to
Hamiltonian Cycle
Definition: Vertex cover is set of vertices that touch all edges in the graph.
Given a 𝑔𝑟𝑎𝑝ℎ 𝐺 and integer k, construct a 𝑔𝑟𝑎𝑝ℎ 𝐺’ such that 𝐺 has a vertex cover of
size k if 𝐺’ has a
𝐻𝑎𝑚𝑖𝑙𝑡𝑜𝑛𝑖𝑎𝑛 𝑐𝑦𝑐𝑙𝑒.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 24
CS-501 Theory of Computation AIST SAGAR
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 25
CS-501 Theory of Computation AIST SAGAR
Path1 has a total length of 24 and Path2 has a total length of 31.
Proof:
First, we have to prove that TSP belongs to NP. If we want to check a tour for credibility, we
check that the tour contains each vertex once. Then we sum the total cost of the edges and finally
we check if the cost is minimum. This can be completed in polynomial time thus TSP belongs
to NP.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 26
CS-501 Theory of Computation AIST SAGAR
Secondly, we prove that TSP is NP-hard. One way to prove this is to show that Hamiltonian
cycle <= TSP (given that the Hamiltonian cycle problem is NP-complete). Assume G = (V, E)
to be an instance of Hamiltonian cycle. An instance of TSP is then constructed. We create the
complete graph = (V, ≤ P G′ E′) where E′ = {(I, j): i, j ∈ V and i ≠ j}. Thus, the cost function is
defined as:
Now suppose that a Hamiltonian cycle h exists in G. It is clear that the cost of each edge in h is
0 in G′ as each edge belongs to E. Therefore, h has a cost of 0 in G′. Thus, if graph G has a
Hamiltonian cycle then graph G′ has a tour of 0 costs.
Conversely, we assume that G’ has a tour h’ of cost at most 0. The cost of edges in E’ are 0 and
1 by definition. So, each edge must have a cost of 0 as the cost of h’ is 0. We conclude that h’
contains only edges in E.
So, we have proven that G has a Hamiltonian cycle if and only if G’ has a tour of cost at most
0. Thus, TSP is NP-complete.
Mr. Rajneesh Pachouri, Asst. Professor, Department of Computer Science & Engineering
Page 27
S.NO. Topic Name Video Link
1 https://www.youtube.com/watch?v=BKhQJP4sa_8
Turing Machine: Techniques for
construction
2 https://www.youtube.com/watch?v=CLLutxGVDY4
Universal Turing machine Multitape