Turing Machine Halting Problem

Input − A Turing machine and an input string w.

Problem − Does the Turing machine finish computing of the string w in a finite number of
steps? The answer must be either yes or no.

Proof − At first, we will assume that such a Turing machine exists to solve this problem and
then we will show it is contradicting itself. We will call this Turing machine as a Halting
machine that produces a ‘yes’ or ‘no’ in a finite amount of time. If the halting machine
finishes in a finite amount of time, the output comes as ‘yes’, otherwise as ‘no’. The
following is the block diagram of a Halting machine −

Now we will design an inverted halting machine (HM)’ as −

● If H returns YES, then loop forever.

● If H returns NO, then halt.

The following is the block diagram of an ‘Inverted halting machine’ −

Further, a machine (HM)2 which input itself is constructed as follows −

● If (HM)2 halts on input, loop forever.

● Else, halt.
Here, we have got a contradiction. Hence, the halting problem is undecidable.

Post Correspondence Problem

The Post Correspondence Problem (PCP), introduced by Emil Post in 1946, is an undecidable
decision problem. The PCP problem over an alphabet ∑ is stated as follows −

Given the following two lists, M and N of non-empty strings over ∑ −

M = (x1, x2, x3,………,xn)

N = (y1, y2, y3,………,yn)

We can say that there is a Post Correspondence Solution, if for some i1,i2,………… ik, where
1 ≤ ij ≤ n, the condition xi1 …….xik = yi1 …….yik satisfies.

Example 1
Find whether the lists

M = (abb, aa, aaa) and N = (bba, aaa, aa)

have a Post Correspondence Solution?

Solution

x1 x2 x3
M abb aa aaa
N bba aaa aa

Here,

x2x1x3 = ‘aaabbaaa’

and y2y1y3 = ‘aaabbaaa’

We can see that

x2x1x3 = y2y1y3

Hence, the solution is i = 2, j = 1, and k = 3.

Example 2
Find whether the lists M = (ab, bab, bbaaa) and N = (a, ba, bab) have a Post
Correspondence Solution?

Solution

x1 x2 x3
M ab bab bbaaa
N A ba bab

In this case, there is no solution because −

| x2x1x3 | ≠ | y2y1y3 | (Lengths are not same)

Hence, it can be said that this Post Correspondence Problem is undecidable.

PDA & Context-Free Grammar

If a grammar G is context-free, we can build an equivalent nondeterministic PDA which
accepts the language that is produced by the context-free grammar G. A parser can be built
for the grammar G.

Also, if P is a pushdown automaton, an equivalent context-free grammar G can be

constructed where

L(G) = L(P)

In the next two topics, we will discuss how to convert from PDA to CFG and vice versa.

Algorithm to find PDA corresponding to a given CFG

Input − A CFG, G = (V, T, P, S)

Output − Equivalent PDA, P = (Q, ∑, S, δ, q0, I, F)

Step 1 − Convert the productions of the CFG into GNF.

Step 2 − The PDA will have only one state {q}.

Step 3 − The start symbol of CFG will be the start symbol in the PDA.

Step 4 − All non-terminals of the CFG will be the stack symbols of the PDA and all the
terminals of the CFG will be the input symbols of the PDA.
Step 5 − For each production in the form A → aX where a is terminal and A, X are
combination of terminal and non-terminals, make a transition δ (q, ε, A) = (q, aX) and
for each terminal symbol make a transition δ (q, a, a) = (q, ε) forall a ϵ ∑

Problem

Construct a PDA from the following CFG.

G = ({S, X}, {a, b}, P, S)

where the productions are −

S → XS | ε , A → aXb | Ab | ab

Solution

Let the equivalent PDA,

P = ({q}, {a, b}, {a, b, X, S}, δ, q, S)

where δ −

δ(q, ε , S) = {(q, XS), (q, ε )}

δ(q, ε , X) = {(q, aXb), (q, Xb), (q, ab)}

δ(q, a, a) = {(q, ε )}

δ(q, 1, 1) = {(q, ε )}

Algorithm to find CFG corresponding to a given PDA

Input − A CFG, G = (V, T, P, S)

Output − Equivalent PDA, P = (Q, ∑, S, δ, q0, I, F) such that the non- terminals of the
grammar G will be {Xwx | w,x∈ Q} and the start state will be Aq0,F.

Step 1 − For every w, x, y, z ∈ Q, m ∈ S and a, b ∈ ∑, if δ (w, a, ε) contains (y, m) and (z, b,

m) contains (x, ε), add the production rule Xwx → a Xyzb in grammar G.

Step 2 − For every w, x, y, z ∈ Q, add the production rule Xwx → XwyXyx in grammar G.

Step 3 − For w ∈ Q, add the production rule Xww → ε in grammar G.

Types of Turing machine

Multi-tape Turing Machine
Multi-tape Turing Machines have multiple tapes where each tape is accessed with a separate
head. Each head can move independently of the other heads. Initially the input is on tape 1
and others are blank. At first, the first tape is occupied by the input and the other tapes are
kept blank. Next, the machine reads consecutive symbols under its heads and the TM prints a
symbol on each tape and moves its heads.

A Multi-tape Turing machine can be formally described as a 6-tuple (Q, X, B, δ, q0, F) where
−
● Q is a finite set of states
● X is the tape alphabet
● B is the blank symbol
● δ is a relation on states and symbols where
δ: Q × Xk → Q × (X × {Left_shift, Right_shift, No_shift })k
where there is k number of tapes
● q0 is the initial state
● F is the set of final states
Note − Every Multi-tape Turing machine has an equivalent single-tape Turing machine.

Multi-track Turing Machine

Multi-track Turing machines, a specific type of Multi-tape Turing machine, contain multiple
tracks but just one tape head reads and writes on all tracks. Here, a single tape head reads n
symbols from n tracks at one step. It accepts recursively enumerable languages like a normal
single-track single-tape Turing Machine accepts.

A Multi-track Turing machine can be formally described as a 6-tuple (Q, X, ∑, δ, q0, F)

where −

● Q is a finite set of states

● X is the tape alphabet
● ∑ is the input alphabet
● δ is a relation on states and symbols where

δ(Qi, [a1, a2, a3,....]) = (Qj, [b1, b2, b3,....], Left_shift or Right_shift)

● q0 is the initial state

● F is the set of final states

Note − For every single-track Turing Machine S, there is an equivalent multi-track Turing
Machine M such that L(S) = L(M).

Non-Deterministic Turing Machine

In a Non-Deterministic Turing Machine, for every state and symbol, there are a group of
actions the TM can have. So, here the transitions are not deterministic. The computation of a
non-deterministic Turing Machine is a tree of configurations that can be reached from the
start configuration.

An input is accepted if there is at least one node of the tree which is an accept configuration,
otherwise it is not accepted. If all branches of the computational tree halt on all inputs, the
non-deterministic Turing Machine is called a Decider and if for some input, all branches are
rejected, the input is also rejected.

A non-deterministic Turing machine can be formally defined as a 6-tuple (Q, X, ∑, δ, q0, B,

F) where −

● Q is a finite set of states

● X is the tape alphabet
● ∑ is the input alphabet
● δ is a transition function;

δ : Q × X → P(Q × X × {Left_shift, Right_shift}).

● q0 is the initial state

● B is the blank symbol
● F is the set of final states

Semi-Infinite Tape Turing Machine

A Turing Machine with a semi-infinite tape has a left end but no right end. The left end is
limited with an end marker.

It is a two-track tape −

● Upper track − It represents the cells to the right of the initial head position.
● Lower track − It represents the cells to the left of the initial head position in reverse
order.

The infinite length input string is initially written on the tape in contiguous tape cells.

The machine starts from the initial state q0 and the head scans from the left end marker ‘End’.
In each step, it reads the symbol on the tape under its head. It writes a new symbol on that tape
cell and then it moves the head either into left or right one tape cell. A transition function
determines the actions to be taken.

It has two special states called accept state and reject state. If at any point of time it enters
into the accepted state, the input is accepted and if it enters into the reject state, the input is
rejected by the TM. In some cases, it continues to run infinitely without being accepted or
rejected for some certain input symbols.

Note − Turing machines with semi-infinite tape are equivalent to standard Turing machines.

Linear Bounded Automata

A linear bounded automaton is a multi-track non-deterministic Turing machine with a tape of
some bounded finite length.

Length = function (Length of the initial input string, constant c)

Here,

Memory information ≤ c × Input information

The computation is restricted to the constant bounded area. The input alphabet contains two
special symbols which serve as left end markers and right end markers which mean the
transitions neither move to the left of the left end marker nor to the right of the right end marker
of the tape.
A linear bounded automaton can be defined as an 8-tuple (Q, X, ∑, q0, ML, MR, δ, F) where
−

● Q is a finite set of states

● X is the tape alphabet
● ∑ is the input alphabet
● q0 is the initial state
● ML is the left end marker
● MR is the right end marker where MR ≠ ML
● δ is a transition function which maps each pair (state, tape symbol) to (state, tape
symbol, Constant ‘c’) where c can be 0 or +1 or -1
● F is the set of final states

A deterministic linear bounded automaton is always context-sensitive and the linear bounded
automaton with empty language is undecidable.

DECIDABLE & UNDECIDABLE PROBLEM

DECIDABLE PROBLEMS

A problem is said to be Decidable if we can always construct a corresponding algorithmthat

can answer the problem correctly. We can intuitively understand Decidable problems by
considering a simple example. Suppose we are asked to compute all the prime numbers in the
range of 1000 to 2000. To find the solution of this problem, we can easily devise an algorithm
that can enumerate all the prime numbers in this range.
Now talking about Decidability in terms of a Turing machine, a problem is said to be a
Decidable problem if there exists a corresponding Turing machine which halts on every input
with an answer- yes or no. It is also important to know that these problems are termed
as Turing Decidable since a Turing machine always halts on every input, accepting or
rejecting it.

Examples
● Equivalence of two regular languages: Given two regular languages, there is an
algorithm and Turing machine to decide whether two regular languages are equal or not.
● Finiteness of regular language: Given a regular language, there is an algorithm and
Turing machine to decide whether regular language is finite or not.
 ● Emptiness of context free language: Given a context free language, there is an
algorithm whether CFL is empty or not.

UndecidableProblems–

The problems for which we can’t construct an algorithm that can answer the problem correctly
in finite time are termed as Undecidable Problems. These problems may be partially decidable
but they will never be decidable. That is there will always be a condition that will lead the
Turing Machine into an infinite loop without providing an answer at all.
We can understand Undecidable Problems intuitively by considering Fermat’s Theorem, a
popular Undecidable Problem which states that no three positive integers a, b and c for any
n>=2 can ever satisfy the equation: a^n + b^n = c^n.
If we feed this problem to a Turing machine to find such a solution which gives a contradiction
then a Turing Machine might run forever, to find the suitable values of n, a, b and c. But we
are always unsure whether a contradiction exists or not and hence we term this problem as
an Undecidable Problem.

Examples

● Ambiguity of context-free languages: Given a context-free language, there is no

Turing machine which will always halt in finite amount of time and give answer whether
language is ambiguous or not.
● Equivalence of two context-free languages: Given two context-free languages, there
is no Turing machine which will always halt in finite amount of time and give answer
whether two context free languages are equal or not.
● Everything or completeness of CFG: Given a CFG and input alphabet, whether CFG
will generate all possible strings of input alphabet (∑*)is undecidable.
● Regularity of CFL, CSL, REC and REC: Given a CFL, CSL, REC or REC,
determining whether this language is regular is undecidable.

CHURCH'S TURING THESIS :-

The Church-Turing thesis states the equivalence between the mathematical concepts of
algorithm or computation and Turing-Machine. It asserts that if some calculation is effectively
carried out by an algorithm, then there exists a Turing machines which will compute that
calculation.

The notion of algorithm, computation, a step-by-step procedure or a defined method to perform

calculations has been used informally and intuitively in mathematics for centuries. However,
attempts to formalize the concept only begun in the beginning of the 20th century. Three major
attempts were made: λ-calculus, recursive functions and Turing Machines. These three formal
concepts were proved to be equivalent; all three define the same class of functions. Hence,
Church-Turing thesis also states that λ-calculus and recursive functions also correspond to the
concept of computability. Since the thesis aims to capture an intuitive concept, namely the
notion of computation, it cannot be formally proven. However, it has gain widely acceptance
in the mathematical and philosophical community.

The thesis has been wrongly attributed to many controversial claims in philosophy, that
although related are not implied in the original thesis. Some examples are:
▪ The universe is equivalent to a Turing Machine and non-computable functions are
physically impossible.[1]
▪ The universe isn’t equivalent to a Turing Machine and incomputable.[2]
▪ The human mind is a Turing Machine, the human mind and/or consciousness are equivalent
to and can be instantiated by a computer. [3]
▪ The human mind isn’t a Turing Machine, the human mind and/or consciousness emerge
due to the existence of incomputable process, such as microtubules performing quantum
process in the brain.[4]

UNIVERSAL TURING MACHINE

● A Turing machine is said to be universal Turing machine if it can accept:
o The input data, and
o An algorithm (description) for computing.
● This is precisely what a general purpose digital computer does. A digital computer
accepts a program written in high level language. Thus, a general purpose Turing
machine will be called a universal Turing machine if it is powerful enough to simulate
the behavior of any digital computer, including any Turing machine itself.
● More precisely, a universal Turing machine can simulate the behavior of an arbitrary
Turing machine over any set of input symbols. Thus, it is possible to create a single
machine that can be used to compute any computable sequence.
If this machine is supposed to be supplied with the tape on the beginning of which is
written the input string of quintuple separated with some special symbol of some
computing machine M, then the universal Turing machine U will compute the same
strings as those by M.
● The model of a Universal Turing machine is considered to be a theoretical breakthrough
that led to the concept of stored programmer computing device.
● Designing a general purpose Turing machine is a more complex task. Once the
transition of Turing machine is defined, the machine is restricted to carrying out one
particular type of computation.
● Digital computers, on the other hands, are general purpose machines that cannot be
considered equivalent to general purpose digital computers until they are designed to
be reprogrammed.
● By modifying our basic model of a Turing machine we can design a universal Turing
machine. The modified Turing machine must have a large number of states for
stimulating even a simple behavior. We modify our basic model by:
o Increase the number of read/write heads
o Increase the number of dimensions of input tape
o Adding a special purpose memory
● All the above modification in the basic model of a Turing machine will almost speed
up the operations of the machine can do.
● A number of ways can be used to explain to show that Turing machines are useful
models of real computers. Anything that can be computed by a real computer can also
be computed by a Turing machine. A Turing machine, for example can simulate any
type of functions used in programming language. Recursion and parameter passing are
some typical examples. A Turing machine can also be used to simplify the statements
of an algorithm.
● A Turing machine is not very capable of handling it in a given finite amount of time.
Also, Turing machines are not designed to receive unbounded input as many real
programmers like word processors, operating system, and other system software.