Foundations of Computer Science Lecture Notes

ENGR 3520 Fall 2013 Monday, Oct 28, 2013

(These notes mostly agree with Kozen’s definitions, except I use a slightly different definition
of configurations that I think is a bit easier to use, although it is entirely equivalent.)

Turing Machines
A Turing machine is like a pushdown automaton except that instead of a stack, we have an
infinite tape that can be written and read by moving the tape head to any cell on the tape.
Moreover, instead of an input string that gets scanned from left to right automatically as
the machine progresses, the input string for a Turing machine is initially provided on the
tape, and the machine must read it explicitly.

Definition. Formally, a (deterministic) Turing machine is a tuple

M = (Q, Σ, Γ, `, , δ, s, t, r)
where

– Q is a finite set of states

– Σ is a finite input alphabet

– Γ is a finite tape alphabet (Σ ⊆ Γ)

– ` is the leftmost marker symbol (` ∈ Γ − Σ)

– is the blank symbol indicating an empty tape cell ( ∈ Γ − Σ)

– δ is the transition function, δ : Q × Γ −→ Q × Γ × {L, R}

– s is the start state

– t is the accept state

– r is the reject state

The transition function takes a state in Q and a symbol in tape alphabet Γ (intuitively, the
symbol written in the tape cell where the tape head is), and tells you the new state to which
the machine transitions, the symbol to write in the cell where the tape head is, and then
whether to move the tape head to the left (L) or to the right (R).

1
The tape is infinite to the right. In its first (leftmost) cell, we assume that the leftmost
marker ` is written. Empty cells are assumed to contain the symbol . We assume that the
machine can never move the tape head to the left of the leftmost marker.

Computation by Turing machine. Here is how a Turing machine M computes when

trying to accept string w = a1 . . . ak ∈ Σ∗ :

(1) Put ` a1 . . . ak on the leftmost cells of the tape.

(2) The tape head is initially on the leftmost cell containing `.

(3) The state is initially s, the start state.

(4) If the machine is in state p and the symbol in the cell where the tape head is is a, and
δ(p, a) = (q, b, d), then the machine writes b in the cell where the tape head is, moves
the tape head in direction d, and moves to state q.

(5) Repeat step (4) until either the state is t (and the machine accepts) or the state is r
(and the machine rejects).

Acceptance. We say machine M accepts w if M reaches the accept state starting with
` w on its input tape.
The language accepted by a Turing machine M is defined by

L(M ) = {w ∈ Σ∗ | M accepts w}

Recognizability. We say a language A is Turing-recognizable (or simply recognizable) if

there exists a Turing machine M such that L(M ) = A.

Examples. Figure 1 gives a simple Turing machine accepting {am bn | m, n ≥ 0}, which
is of course a regular language. As usual, we simply draw the diagram of state transitions,
where a transition of the form δ(p, a) = (q, b, d) is drawn as an arrow between states p and
q labelled by a/b, d. The tape alphabet is given when unclear.
Figure 2 gives an already more complex Turing machine accepting {an bn | n ≥ 0}, which is
a context-free language. The machine works in two phases. First, it scans the input string
from left to right to ensure it is of the form am bn for some m, n. If not, then it goes to a
sink state. If it is of that form, then the tape head is rewound to the left, and finds first an
a and then a b, replacing both by a new tape symbol X. The tape head is rewound to the
left again, and another a and another b are crossed out, skipping over previous crossed out
symbols. If a b is found when an a was looked for, or no b found after an a is found, then the
machine transitions to a sink state, otherwise, it accepts. The tape alphabet is {`, , a, b, X}.

2
 a/a,R
b/b,R
⌴/⌴,R
⊢/⊢,R

a/a,R
⊢/⊢,R
a/a,R
⊢/⊢,R

b/b,R b/b,R
s

⌴/⌴,R ⌴/⌴,R

a/a,R
b/b,R
⌴/⌴,R
⊢/⊢,R

Figure 1: Turing machine accepting {am bn | m, n ≥ 0}

Two things to note about these Turing machines. First off, since the transition function is a
function, it needs to be defined for every combination of state and tape symbol. This means,
in particular, that the accept state needs to have transitions out of it for every tape symbol.
The convention is that those transitions just loop back to the accept state itself, making the
accept state a sink state. (The same thing holds true for the reject state as well.)
Second off, I did not give a reject state for those machines. Part of the reason is pedagogical:
I want to show that we can define and talk about languages accepted by Turing machines
without making reference to reject states. We’ll see reject states next lecture. Now, strictly
speaking, the description of a Turing machine says that there has to a reject state. When
one is not explicitly given, say, when describing a Turing machine via a diagram, we can
just assume that there is a reject state that is not connected to any other state, and whose
transitions simply loop back to itself.

3
 a/a,R
b/b,R
⌴/⌴,R
⊢/⊢,R
X/X,R

a/a,R X/X,R
⊢/⊢,R
a/a,R
⊢/⊢,R
X/X,R
b/b,R
s b/b,R

X/X,R a/a,R
⌴/⌴,R ⌴/⌴,R b/b,R
X/X,R
⌴/⌴,R
⊢/⊢,R

t
⊢/⊢,R ⌴/⌴,R

a/a,L
b/b,L b/b,R
X/X,L a/X,R a/a,R
⊢/⊢,R
⌴/⌴,L b/b,R
b/X,R X/X,R
⌴/⌴,R
⊢/⊢,R

⌴/⌴,R
⊢/⊢,R

a/a,R
X/X,R

Figure 2: Turing machine accepting {an bn | n ≥ 0}

It is an easy exercise to modify the Turing machine in Figure 2 to accept the language
{an bn cn | n ≥ 0}, which is not context-free. The resulting Turing machine is in Figure 3.

4
 a/a,R
b/b,R
c/c,R
⌴/⌴,R
⊢/⊢,R
X/X,R

c/c,R
X/X,R a/a,R
a/a,R
b/b,R
⊢/⊢,R
⊢/⊢,R
X/X,R
b/b,R a/a,R
⊢/⊢,R
X/X,R c/c,R
s b/b,R
c/c,R

a/a,R
X/X,R b/b,R
⌴/⌴,R ⌴/⌴,R ⌴/⌴,R c/c,R
X/X,R
⌴/⌴,R
⊢/⊢,R
t
⊢/⊢,R ⌴/⌴,R

a/a,L
b/b,L b/b,R
a/X,R a/a,R
c/c,L c/c,R b/b,R
X/X,L ⊢/⊢,R c/c,R
⌴/⌴,L
a/a,R X/X,R
X/X,R ⌴/⌴,R
c/c,R
⊢/⊢,R
⌴/⌴,R
⊢/⊢,R

b/X,R
c/X,R
a/a,R
⌴/⌴,R
⊢/⊢,R

b/b,R
X/X,R

Figure 3: Turing machine accepting {an bn cn | n ≥ 0}

5
 Foundations of Computer Science Lecture Notes
ENGR 3520 Fall 2013 Thursday, Oct 31, 2013

Rejection. We say machine M rejects w if M reaches the reject state starting with ` w
on its input tape.
Machine M halts on input w if M either accepts w or rejects w.
We say machine M is total if it halts on every input string in Σ∗ .

Decidability. We say a language A is Turing-decidable (or simply decidable) if there exists

a total Turing machine M such that L(M ) = A.
Clearly, every decidable language is also recognizable, since total Turing machines are just
a special class of Turing machines.
Recasting the definition, a language is decidable if and only if there a Turing machine M
that accepts every string w ∈ L(M ) and that rejects every string w 6∈ L(M ).
We say a Turing machine M decides language A if M is total and M accepts A.
It is easy to see that every regular language is decidable; it suffices to show that DFAs
can be simulated by Turing machines that always halt.1 It is slightly less easy to see that
every context-free language is decidable; here it suffices to show that a Turing machine can
simulate a context-free grammar.2
Figure 4 gives a total Turing machine deciding {an bn | n ≥ 0}. It basically amounts to
taking the Turing machine in Figure 2 and replacing the sink states by a single reject state.
We can similarly exhibit a total Turing machine deciding {an bn cn | n ≥ 0}. This shows that
the class of decidable languages is strictly larger than the class of context-free languages.
As we shall see, there are languages that are recognizable but not decidable. And similarly,
languages that are not even recognizable. Given how general and expressive Turing machines
are, this is somewhat surprising.

Pseudocode Descriptions of Turing Machines. Giving complete descriptions of Tur-

ing machines becomes painful for anything but the simplest of machines. Therefore, we will
generally resort to a pseudocode description of the behavior of Turing machines, focusing on
tape head movement, and symbol replacement. For example, here is a reasonable description
of a total version of the Turing machine accepting {an bn cn | n ≥ 0} from Figure 3.
1
We will see this in Homework 6.
2
We will probably see this in Homework 7.

6
 On input w:
1. Scan tape from left to right, checking that as follow bs follow cs. Reject if not.
2. Move tape head back to leftmost position.
3. Scan from left to right, replacing the first a encountered with X,
then the first b encountered with X, then the first c encountered with X.
4. If no a, b, or c encountered, accept.
5. If any of a, b, or c is not encountered, reject.
6. Go back to step 2.
It must be the case that every step in the pseudocode description of a Turing machine should
be easily translatable into a set of transitions between states.

Configurations. A configuration is a snapshot of the execution of a Turing machine. To

describe the Turing machine at any point in its execution, we need to give: the content of
the tape, the position of the tape head, and the current state of the machine.
This information can be represented by a triple (u, q, v), where the content of the tape is the
string uv,3 the tape head is currently pointing to the tape cell at position |u|, and thus the
first symbol of v is currently in the cell where the tape head is, and the current state of the
machine is q. We do not allow v to be ; if ever we have a configuration where v is , we
rewrite it as a configuration where v = . (This captures the fact that there is an infinite
supply of s on the tape.)
Thus, a configuration is a tuple in Γ∗ × Q × Γ+ .
Fix a Turing machine M = (Q, Σ, Γ, `, , δ, s, t, r).
A starting configuration for M is a configuration of the form (, s, ` w) for some input string
w.
An accepting configuration for M is a configuration of the form (u, t, v) for some u, v.
A rejecting configuration for M is a configuration of the form (u, r, v) for some u, v.
A halting configuration is a configuration that is either accepting or rejecting.
1
We define a step relation between configurations, written C −→ C 0 , that describes how
M
configuration evolve as the Turing machine computes and transitions between states. The
step relation is defined by the following rules:
1
(u, p, av) −→ (ub, q, v) if δ(p, a) = (q, b, R)
M
1
(uc, p, av) −→ (u, q, cbv) if δ(p, a) = (q, b, L)
M

(It seems that the step relation is not defined for a configuration of the form (u, p, v) with u
3
We only need to describe the content of the tape up to the first blank space to the right of which there
are only blank spaces, and there can only be finitely many cells that have been rewritten by the Turing
machine during any execution, so uv is a finite string.

7
 a/a,R
b/b,R
a/a,R X/X,R ⌴/⌴,R
⊢/⊢,R ⊢/⊢,R
X/X,R
r
a/a,R
b/b,R ⊢/⊢,R
X/X,R
s q1 b/b,R

b/b,R
X/X,R ⊢/⊢,R a/a,R
⌴/⌴,R ⌴/⌴,R b/b,R
X/X,R
⌴/⌴,R
⊢/⊢,R

q2 q3 t
⊢/⊢,R ⌴/⌴,R

a/a,L
b/b,L
X/X,L a/X,R
⌴/⌴,L
b/X,R

q4 ⌴/⌴,R
⊢/⊢,R

a/a,R
X/X,R

Figure 4: Total Turing machine deciding {an bn | n ≥ 0}

empty or v empty. If v is empty, then we saw above that we replace v by , so we’re good
to go. If u is empty, then the tape head is at the way beginning of the tape, with ` written
in the cell pointed to by the tape head, and we saw that we always assumed that the Turing
machine always moves right when ` is written in the cell pointed to by the tape head.)
∗ ∗ ∗ ∗
We define C −→ C 0 if either C = C 0 or there exists C1 , . . . , Ck such that C −→ C1 −→ . . . −→
M M M M
∗
Ck −→ C 0 .
M
∗
We can now define acceptance formally: M accepts w if (, s, ` w) −→ Cacc for some
M
accepting configuration Cacc .
∗
Similarly, M rejects w if (, s, ` w) −→ Crej for some rejecting configuration Crej .
M

Example. As an example, consider the total Turing machine in Figure 4, which is a total
variant of the Turing machine in Figure 2.

8
Here is the sequence of configurations showing how the machine accepts aabb:
1
(, s, ` aabb) −→ ( `, s, aabb)
M
1
−→ ( ` a, s, abb)
M
1
−→ ( ` aa, s, bb)
M
1
−→ ( ` aab, q1 , b)
M
1
−→ ( ` aabb, q1 , )
M
1
−→ ( ` aabb , q2 , )
M
1
−→ ( ` aabb, q2 , )
M
1
−→ ( ` aab, q2 , b )
M
1
−→ ( ` aa, q2 , bb )
M
1
−→ ( ` a, q2 , abb )
M
1
−→ ( `, q2 , abb )
M
1
−→ ( `, q2 , aabb )
M
1
−→ (, q2 , ` aabb )
M
1
−→ ( `, q3 , aabb )
M
1
−→ ( ` X, q4 , abb )
M
1
−→ ( ` Xa, q4 , bb )
M
1
−→ ( ` XaX, q2 , b )
M
1
−→ ( ` Xa, q2 , Xb )
M
1
−→ ( ` X, q2 , aXb )
M
1
−→ ( `, q2 , XaXb )
M
1
−→ (, q2 , ` XaXb )
M
1
−→ ( `, q3 , XaXb )
M
1
−→ ( ` X, q3 , aXb )
M

9
 1
−→ ( ` XX, q4 , Xb )
M
1
−→ ( ` XXX, q4 , b )
M
1
−→ ( ` XXXX, q2 , )
M
1
−→ ( ` XXX, q2 , X )
M
1
−→ ( ` XX, q2 , XX )
M
1
−→ ( ` X, q2 , XXX )
M
1
−→ ( `, q2 , XXXX )
M
1
−→ (, q2 , ` XXXX )
M
1
−→ ( `, q3 , XXXX )
M
1
−→ ( ` X, q3 , XXX )
M
1
−→ ( ` XX, q3 , XX )
M
1
−→ ( ` XXX, q3 , X )
M
1
−→ ( ` XXXX, q3 , )
M
1
−→ ( ` XXXX , t, )
M

and this last configuration ( ` XXXX , t, ) is an accepting configuration, and thus w is ac-
cepted.

10
 Foundations of Computer Science Lecture Notes
ENGR 3520 Fall 2013 Monday, Nov 4, 2013

Turing machines are simple. Despite their simplicity, they are powerful. But while their
simplicity is useful for proving things about them, it is painful when the time comes to
design Turing machines to recognize or decide specific languages.
Variants of Turing machines exist that are simpler to use, but that still lead to the same
class of recognizable (and decidable) languages.

Multitape Turing Machines. For instance, while a Turing machine has a single tape,
we can imagine working with a Turing machine with multiple tapes. Having multiple tapes
(with independent tape heads) means that we can use some of those tapes as temporary
storage, or scratch pad to perform calculations, and so on. Having more than one tape is
handy.
We can define multitape Turing machines easily enough. A multitape Turing machine (with
k tapes) is a tuple M = (Q, Σ, Γ, `, , δ, s, t, r) defined exactly as a one-tape Turing machine,
except that the transition relation δ has the form

δ : Q × Γk −→ Q × Γk × {L, R}k .

Intuitively, δ(p, ha1 , . . . , ak i) = (q, hb1 , . . . , bk i, hd1 , . . . , dk i) says that when in state p and
when ai is on tape i under tape i0 s tape head, then M can transition to state q, writing bi
on tape i under its tape head, and moving each tape head in direction di . As for Turing
machines, we assume that every tape has a leftmost marker in its leftmost cell, and that the
machine cannot move the tape head to the left when on the leftmost marker.
This can be formalized using the notion of configuration, as with standard Turing machines.
A k-tape configuration is a tuple (u1 , q, v1 , u2 , v2 , u3 , v3 , . . . , uk , vk ) where q is a state to the
multitape Turing machine and ui , vi are all strings over the tape alphabet of the machine.
As usual, when any of the vi is , we replace it with a to capture the fact that tapes are
infinite to the right and filled with blanks. The starting configuration of the machine with
input w is simply (, s, ` w, , vdash, , `, . . . , , `), where s is the start state of the machine; a
configuration is accepting (resp., rejecting) if the state is the accept (resp., reject) state of
1
the machine. The step relation C −→ C 0 is defined exactly as expected:
M

1
(u1 , p, a1 v1 , u2 , a2 v2 , . . . , uk , ak vk ) −→ (u01 , q, v10 , u02 , v20 , . . . , u0k , vk0 )
M
if δ(p, ha1 , . . . , ak i) = (q, hb1 , . . . , bk i, hd1 , . . . , dk i)
and for all i, if di = R then u0i = ui bi and vi0 = vi
and if di = L and ui = uc, then u0i = u and vi0 = cbi vi

11
 ∗
C −→ C 0 is defined as for Turing machines, and M accepting and rejecting a string w is
M
∗
defined as for Turing machines using the −→ relation. The language of a multitape Turing
M
machine M is just the set of strings accepted by M . A multitape Turing machine is total if
it halts on all inputs.
We say a language is recognizable by a multitape Turing machine if there is a multitape
Turing machine that accepts it. It is decidable by a multitape Turing machine if there is a
total multitape Turing machine that accepts it.
Multitape Turing machines, while convenient, do not give us more recognizable or decidable
languages.
Theorem: A language is recognizable (resp., decidable) by a multitape Turing machine if
and only if it is recognizable (resp., decidable).
We first prove the reverse direction: if a language is recognizable, it is recognizable by a
multitape Turing machine. If a language is recognizable, there is a Turing machine that
accepts it. A Turing machine is just a multitape Turing machine with a single tape (you can
check the definitions above agree in that case), and so there is a multitape Turing machine
(with one tape) that accepts it. Same things if the language is decidable.
The forward direction is more interesting. Suppose a language A is recognizable by a mul-
titape Turing machine, say Mmt , with k tapes. To show it is recognizable, we need to show
that there is a Turing machine MA that accepts A. We build this Turing machine MA by
essentially simulating what Mmt is doing, but using a single tape. The idea is to put all the
tapes that Mmt would use on a single tape, separated by a special tape symbol # that marks
where each simulated tape ends and a new simulated tape begins. We indicate where each
tape head is by using marked tape symbols of the form b a, adding those marked symbols to
the tape alphabet.
When MA runs, it simulates every step of Turing machine Mmt : whenever Mmt would take
a transition in state q, MA takes a sequence of transitions from a state q corresponding to
q that first determine what transition Mmt would make, updates the tapes accordingly, and
then transitions to a new state q 0 that corresponds to the state q 0 that Mmt transitions to.
Given Mmt , MA is defined as follows:
On input w:
1. Rewrite ` a1 . . . an on the tape into ` b# a1 . . . an b
# b # . . . # (k + 1 #s in total)
# b
2. Simulate a move of Mmt :
a. Scan tape from left to right, noting the marked symbols until the (k + 1)th #
b. scan tape from left to right, replacing marked symbols, and moving the mark
left or right according to Mmt (treat # as `)
c. if the mark moves right onto a #, shift whole tape right from that position
and write b in the cell
3. If Mmt accepts, accept; if Mmt rejects, reject
4. Go to step 2

12
Observe that MA is total exactly when Mmt is total; this means that the construction in
fact shows that if A is recognizable (resp., decidable) by a multitape Turing machine, it is
recognizable (resp., decidable).

Nondeterministic Turing Machines. Another variant of Turing machines that comes

in handy are nondeterministic Turing machines, that bear a similar relationship to deter-
ministic Turing machines that NFAs bear to DFAs. A nondeterministic Turing machine can
have more than one transition available at a state for a given symbol at the tape head.
We can define nondeterministic Turing machines easily enough. A nondeterministic Turing
machine is a tuple M = (Q, Σ, Γ, `, , ∆, s, t, r) defined exactly as a deterministic Turing
machine, except that the transition relation ∆ has the form

∆ : Q × Γ −→ 2Q×Γ×{L,R} .

Intuitively, δ(p, a) returns a set of possible transitions, each of the form (q, b, d) for a state
q, a symbol b to write on the tape at the tape head, and direction in which to move the
tape. Again, just like NFAs: we don’t just have a single transition, we have multiple possible
transitions. As usual, we assume that the tape has a leftmost marker in its leftmost cell,
and that the machine cannot move the tape head to the left when on the leftmost marker.
We define how nondeterministic Turing machines compute using configurations. We use the
same configurations as for deterministic Turing machines, that is, tuples (u, q, v) where u, v
are strings over Γ, and q is a state in Q.
The starting configuration of the machine with input w is (, s, ` w), where s is the start
state of the machine; a configuration is accepting (resp., rejecting) if the state is the accept
(resp., reject) state of the machine.
1
The step relation C −→ C 0 between configurations for nondeterministic Turing machines is
M
where the main difference with deterministic Turing machines lies. The step relation is such
that a configuration may lead to many next configurations, capturing the nondeterminism
of the machine.
1
(u, p, av) −→ (ub, q, v) if (q, b, R) ∈ δ(p, a)
M
1
(uc, p, av) −→ (u, q, cbv) if (q, b, L) ∈ δ(p, a)
M

∗ 1 1 1 1
We define C −→ C 0 if C = C 0 or there exists C1 , . . . , Ck such that C −→ C1 −→ C2 −→ . . . −→
M M M M M
1
Ck −→ C 0 .
M
∗
We say nondeterministic Turing machine M accepts string w if (, s, ` w) −→ Cacc for some
M
accepting configuration Cacc . We say nonterministic Turing machine M rejects a string
∗
w if whenever (, s, ` w) −→ C and C is a halting configuration, then C is a rejecting
M

13
configuration. A nondeterministic Turing machine is total if it always halts on all inputs
(i.e., there is no infinite path in the configuration tree, for any input string). The language
of a nondeterministic Turing machine M is the set of strings accepted by M .
We say a language is recognizable by a nondeterministic Turing machine if there is a non-
deterministic Turing machine that accepts it. It is decidable by a nondeterministic Turing
machine if there is a total nondeterministic Turing machine that accepts it.
Nondeterministic Turing machines also do not give us more recognizable or decidable lan-
guages.
Theorem: A language is recognizable (resp., decidable) by a nondeterministic Turing ma-
chine if and only if it is recognizable (resp., decidable).
The reverse direction is easy: deterministic Turing machines are just special cases of nonde-
terministic Turing machines, in which there is, for every state and every tape symbol, exactly
one transition enabled. Thus, if A is recognizable, it is recognizable by a nondeterministic
Turing machine. Similarly if A is decidable.
The forward direction is more interesting. Suppose a language A is recognizable by a non-
deterministic Turing machine, say Mnd . To show it is recognizable, we need to show that
there is a Turing machine MA that accepts A. In fact, we are going to show that there
is a deterministic Turing machine with 3 tapes that accepts A. (Since multitape Turing
machines accept exactly the same class of languages as turing machines, we get our result.)
We build this 3-tape Turing machine MA by essentially doing a breadth-first search through
the configuration tree of Mnd . The first tape will hold the original input string. The second
tape will hold the queue of configurations through which we search. The third tape will hold
the current configuration, which we will use to simulate a step of Mnd .
Given Mnd with states Q and tape alphabet Γ, MA has a tape alphabet that includes Γ and
Q (so that states of Mnd are tape symbols of MA , so we can write down a configuration onto
the tape) as well as special symbols # and / and is defined as follows:
On input w:
1. Rewrite ` a1 . . . ak into ` / / s / a1 . . . ak on the first tape, where s is the
start state of Mnd (/ separates configuration components)
2. Copy the first tape to the second tape
3. If the second tape is empty, reject
4. Move the last string on the second tape to the third tape, up to the # symbol,
erasing the string and the # symbol
5. If the string on the third tape is an accepting configuration, accept
6. If the string on the third tape is a rejecting configuration, go to step 8
7. Simulate a move of Mnd :
a. Scan third tape from left to right, finding the state in the configuration

14
 b. For every possible nondeterministic transition of Mnd in the configuration
on the third tape, write the result of taking the transition as a configuration
at the beginning of the second tape, shifting the tape to the right to make
room, and write a # after the configuration
8. Erase the third tape
9. Go to step 3
Observe once again that if the original Mnd is total, then MA is total. Thus, this construction
also shows that if A is decidable by a nondeterministic Turing machine, it is decidable.

Church-Turing Thesis. What the above shows is that the definition of decidable lan-
guages is quite robust: extensions to Turing machines do not add any expressive power to
Turing machines. Intuitively, standard Turing machines are powerful enough to be able to
simulate any variant of Turing machines.
In fact, Turing machines are powerful enough, as far as we know, to simulate any model
of computation, or any programming language. This thesis is known as the Church-Turing
thesis:
The Church-Turing Thesis: Any feasible model of computation can be simulated by Turing
machines.
The notion feasible is kept vague, partly because it cannot really be defined. But it captures
the idea of a model of computation that is, for instance, physically realizable—it does not
depend on the ability to perform an infinite amount of computation in a finite amount of
time, or requiring an infinite amount of space.
Because feasible cannot be defined, the Church-Turing thesis is not a theorem. Nevertheless,
we have a lot of evidence that the thesis is true.
While it may seem surprising that a Turing machine can in fact simulate any model of
computation, it should not be surprising to us that have grown up in the computer age
that computers can simulate any model of computation. It remains to show that Turing
machines can simulate computers. But a computer is really just a CPU with a bunch of
attendant hardward to make it interact with the world, and a CPU is really just a sort of
Turing machines. Putting it the other way around, it is not difficult to imagine simulating
a CPU with a Turing machine: the alphabet of the Turing machine includes 0 and 1, which
are the basic values that a CPU works with; the registers of the CPU can live on the tape;
the finite number of gates of the CPU can be modeled by the finite control of the Turing
machine; and the memory that the CPU has access to, which is finite, can be against stored
on the tape. While a CPU addresses memory by indexing into it, the Turing machine would
have to move the tape head to the appropriate cell corresponding to the memory location,
but the difference is merely one of efficiency.4
4
The Church-Turing thesis never claimed that Turing machines can simulate any model of computation
efficiently, just that they can simulate it.

15
The Church-Turing thesis makes the distinctions between decidable languages and undecid-
ables languages relevant for the casual programmer. Recall that languages over an alphabet
Σ are just functions from Σ∗ to the Booleans. A language A is decidable if it can be accepted
by a total Turing machine; in other words, a function A from Σ∗ to the Booleans is decidable
if you can come up with a total Turing machines that says yes to a string w exactly when the
function A maps w to true. And equivalently, a language is undecidable if there can be no
total Turing machine that accepts it. But because of the Church-Turing thesis, if a language
is undecidable, then not only can you have no total Turing machine that accepts it, you can
also have no program in any of your favorite programming languages that implements the
function corresponding to the language: if you could, then you could also write a Turing
machine that simulates that program, and this would contradict undecidability. Thus, if a
language is undecidable, there can be no program that implements the corresponding func-
tion, period. The notion of decidability, even though it seems like a technical notion having
to do with Turing machines only, in fact impacts programming in general. Being able to
determine what languages are decidable and which aren’t determines which problems can be
solved using computer programs, and which can’t.

16