Introduction to the Theory

of Computation

Part II: Computability

Theory
3. The Church-Turing
Thesis

3.1 Turing Machines

3.2 Variants of Turing Machines
 Multitape Turing Machines
 Nondeterministic Turing Machines
 Enumerators
 Equivalence with other models
3.3 The Definition of Algorithm
Turing Machines
input tape
0 1 1 0 0 1 1 1 0 1 0 0 …
finite
control read/write
head
q0
New capabilities:
infinite tape
can read OR write to tape
read/write head can move left and right
Turing Machines

Informal description:
 input written on left-most squares of tape
 rest of squares are blank
 at each point, take a step determined by
 current symbol being read
 current state of finite control
 a step consists of
 writing new symbol
 moving read/write head left or right
 changing state
Turing Machine Diagrams
a → b,L
a→R
start state states
b→L (1 accept
_→R
qreject qaccept + 1 reject)
b → a,R

transition label: (tape symbol read → “_” means

tape symbol written, direction moved) blank tape
 a → R means “read a, move right” square
 a → L means “read a, move left”
 a → b, R means “read a, write b, move right
Example Turing Machine

language L = {w#w : w  {0,1}*}

input tape
0 1 # 0 1 …
finite
control
q0
read/write
head
Example TM Diagram
0,1 → R 0,1 → R 0,1,# → L 0,1 → R x→R
#→R _→L _→R #→R
q1 q3 q5 q7 q9
0 → _,R
0 → x, R 0 → x, L
0,1,x,# → L x→R x→R
q0 _→R #→R _→R
q11 q12 q13 qaccept

#→R
1 → x, R 1 → x, L
1 → _,R
q2 q4 q6 q8 q10
#→R _→L _→R #→R
0,1 → R 0,1 → R 0,1,# → L 0,1 → R x→R
TM Formal Definition

A TM is a 7-tuple
(Q, Σ, , δ, q0, qaccept, qreject) where:
 Q is a finite set called the states
 Σ is a finite set called the input alphabet
  is a finite set called the tape alphabet,   Σ{‘_’}
 δ: Q x  → Q x  x {L, R} is a function called the
transition function
 q0 is an element of Q called the start state
 qaccept, qreject are the accept and reject states
Example TM Operation
program
for “binary
# 0 1 start successor”
# 0 1 start q σ δ(q,σ)
# 0 1 start start 0 (start, 0, R)
start 1 (start, 1, R)
# 0 1 start start _ (t, _, L)

# 0 1 start # (start, #, R)
t
t 0 (accept, 1, -)
# 0 0 t t 1 (t, 0, L)
t # (accept, #, R)
# 1 0 accept
TM Configurations

At every step in a computation, a TM is in a

configuration determined by:
 the current tape contents
 the current state
 the current head location
next step completely determined by current
configuration
shorthand: string uqv with u,v  *, q  Q
TM Configurations

configuration C1 yields configuration C2 if

TM can legally move from C1 to C2 in 1 step
notation: C1  C2
also: “yields in 1 step” notation: C1 1 C2
“yields in k steps” notation: C1 k C2
if there exists configurations D1,D2,…Dk-1 for
which C1  D1  D2  …  Dk-1  C2
also: “yields in some # of steps” (C1 * C2)
TM Configurations

Formal definition of “yields”: u,v  *

a,b,c  
uaqibv  uqjacv
q i, q j  Q
if δ(qi, b) = (qj, c, L), and
uaqibv  uacqjv
if δ(qi, b) = (qj, c, R)
two special cases:
left end: qibv  qjcv if δ(qi, b) = (qj, c, L)
right end: uaqi same as uaqi_
TM Acceptance
start configuration: q0w (w is input)
accepting config.: any config. with state qaccept
rejecting config.: any config. with state qreject
accepting config. and rejecting config. are halting
config.
TM M accepts input w if there exist configurations C1,
C2, …, Ck
 C1 is start configuration of M on input w
 Ci  Ci+1 for i = 1, 2, 3, …, k-1
 Ck is an accepting configuration
Deciding and Recognizing
• accept

input machine • reject

• loop forever

TM M:
 L(M) is the language it recognizes
 if M rejects every x  L(M) it decides L
 set of languages recognized by some TM is called
Turing-recognizable or recursively enumerable (RE)
 set of languages decided by some TM is called
Turing-decidable or decidable or recursive
Classes of Languages

decidable
all languages
regular
languages

context free
languages RE

We know: regular  CFL (proper containment)

CFL  decidable?
decidable  RE  all languages?
Multitape TMs
A useful variant: k-tape TM
input tape
finite 0 1 1 0 0 1 1 1 0 1 0 0 …
control k read/write
q0 heads

0 1 1 0 0 1 …
0 1 1 0 0 1 1 1 0 1 0 0 …
k-1 “work
…
tapes”
0 …
Multitape TMs

Informal description of k-tape TM:

 input written on left-most squares of tape #1
 rest of squares are blank on all tapes
 at each point, take a step determined by
 current k symbols being read on k tapes
 current state of finite control
 a step consists of
 writing k new symbols on k tapes
 moving each of k read/write heads left or right
 changing state
Multitape TM formal
definition

A TM is a 7-tuple
(Q, Σ, , δ, q0, qaccept, qreject) where:
 everything is the same as a TM except the transition
function:
δ: Q x k → Q x k x {L, R}k

δ(qi, a1,a2,…,ak) = (qj, b1,b2,…,bk, L, R,…, L) =

“in state qi, reading a1,a2,…,ak on k tapes, move to state
qj, write b1,b2,…,bk on k tapes, move L, R on k tapes
as specified.”
Multitape TMs

Theorem: every k-tape TM has an

equivalent single-tape TM.

Proof:
Idea: simulate k-tape TM on a 1-tape TM.
Multitape TMs

simulation of k-tape TM by single-tape TM:

a b a b ...
• add new symbol
(input tape)
x for each old x
a a ...
• marks location of
b b c d . . . “virtual heads”

# a b a b # a a # b b c d # ...
Multitape TMs
a b a b . . . Repeat:
• scan tape, remembering the symbols
a a ... under each virtual head in the state
(how many new states needed?)

... • make changes to reflect 1 step of M

b b c d
• if hit #, shift to right to make room
if M halts, erase all but 1st string

# a b a b # a a # b b c d # ...
Nondeterministic TMs

An important variant: nondeterministic TM

informally, several possible next
configurations at each step
formally, A NTM is a 7-tuple
(Q, Σ, , δ, q0, qaccept, qreject) where:
everything is the same as a TM except the
transition function:
δ: Q x  → (Q x  x {L, R})
NTM acceptance

start configuration: q0w (w is input)

accepting config.: any config.with state qaccept
rejecting config.: any config. with state qreject
TM M accepts input w if there exist
configurations C1, C2, …, Ck
C1 is start configuration of M on input w
Ci  Ci+1 for i = 1, 2, 3, …, k-1
Ck is an accepting configuration
Nondeterministic TMs

Theorem: every NTM has an equivalent

(deterministic) TM.

Proof:
Idea: simulate NTM with a deterministic TM
Nondeterministic TMs
Simulating NTM M with a deterministic TM:
Cstart
• computations of M are a tree
• nodes are configurations
• fanout is b = maximum
number of choices in transition
rej acc function
• leaves are accept/reject
configurations.
Nondeterministic TMs

Simulating NTM M with a deterministic TM:

idea: breadth-first search of tree
if M accepts: we will encounter accepting
leaf and accept
if M rejects: we will encounter all rejecting
leaves, finish traversal of tree, and reject
if M does not halt on some branch: we will
not halt as that branch is infinite…
Nondeterministic TMs

Simulating NTM M with a deterministic TM:

use a 3 tape TM:
 tape 1: input tape (read-only)
 tape 2: simulation tape (copy of M’s tape at point
corresponding to some node in the tree)
 tape 3: which node of the tree we are exploring
(string in {1,2,…b}*)
Initially, tape 1 has input, others blank
Nondeterministic TMs
Simulating NTM M with a deterministic TM:
 STEP 1: copy tape 1 to tape 2
 STEP 2: use tape 2 to simulate M on one branch of its
nondeterministic computation, consult the string on
tape 3 to determine which choice to make at each step
 if encounter blank, or a number larger than the number of
choices available at this step, abort, go to STEP 3
 if get to a rejecting configuration, go to STEP 3
 if get to an accepting configuration, ACCEPT
 STEP 3: replace tape 3 with lexicographically next
string and go to STEP 1
Recursive Enumerability

Why is “Turing-recognizable” called RE?

Definition: a language L  Σ* is recursively
enumerable if there exists a TM (an “enumerator”)
that writes on its output tape
#x1#x2#x3#...
and L = {x1, x2, x3, …}.

The output may be infinite

Recursive Enumerability

Theorem: A language is Turing-recognizable iff

some enumerator enumerates it.

Proof:
() Let E be the enumerator. On input w:
 Simulate E. Compare each string it outputs with w.
 If w matches a string output by E, accept.
Recursive Enumerability

Theorem: A language is Turing-recognizable iff

some enumerator enumerates it.
Proof:
() Let M recognize language L  Σ*.
 let s1, s2, s3, … be enumeration of Σ* in lexicographic
order.
 for i = 1,2,3,4,…
 simulate M for i steps on s1, s2, s3, …, si
 if any simulation accepts, print out that sj
Church-Turing Thesis

many other models of computation

 we saw multitape TM, nondeterministic TM
 others don’t resemble TM at all
 common features:
 unrestricted access to unlimited memory
 finite amount of work in a single step
 every single one can be simulated by TM
 many are equivalent to a TM
The underlying class of algorithms that different
computational models describe is unique!
Church-Turing Thesis

the intuitive notion of an algorithm, or an

effective or mechanical method, M:
 M is set out in terms of a finite number of
instructions;
 M will, if carried out without error, produce the
desired result in a finite number of steps;
 M can (in practice or in principle) be carried out by a
human being unaided by any machinery save paper
and pencil;
 M demands no insight or ingenuity on the part of the
human being carrying it out;
Church-Turing Thesis

Church-Turing thesis captures the intuitive

notion of algorithms precisely
Turing’s thesis: every effective computation
can be carried out by a Turing machine.
Church: -calculus
They turned out to be equivalent
Church-Turing Thesis

There are well-defined mathematical problems

that cannot be solved by effective methods.
E.g. Hilbert’s tenth problem: determination of
the solvability of a Diophantine equation.
 Given a Diophantine equation with any number of
unknown quantities and with rational integral
numerical coefficients: To devise a process according
to which it can be determined by a finite number of
operations whether the equation is solvable in
rational integers.
Hilbertian Dream

1900, Hilbert --- mathematicians should seek to

express mathematics in the form of a
consistent, complete and decidable formal
system.
1931, Goedel --- there can be no consistent and
complete formal system of arithmetic
(incompleteness theorem).
1936, Church and Turing --- no consistent
formal system of arithmetic is decidable.
Alan Turing (1912-1954)
1912, born
1931, King’s College, Cambridge
1935, a fellow of King’s College
1936-1938, Ph.D. in Princeton with Church
1938, back to King’s College
1939, Bletchley Park
1945, National Physical Laboratory
1948, Royal Society Computing Machine Laboratory at
Manchester University
1954, died
High-Level Description

Convince yourself that the following types

of operations are easy to implement as
part of TM “program”
(but perhaps tedious to write out…)
copying
moving
incrementing/decrementing
arithmetic operations +, -, *, /
Encoding of Input

the input to a TM is always a string in Σ*

we must encode our input as such a string
examples:
 tuples separated by #: #x#y#z
 0/1 matrix given by: #n#x# where x  {0,1}n2
 graph in adjacency list format
any reasonable encoding is OK
emphasize “encoding of X” by writing <X>
Assignment 6
1 、 Consider the (deterministic) Turing machine M
M = ({q0,q1,q2}, {a,b}, {a,b,_}, , q0, q2, qreject)
which has exactly four transitions defined in it as below.
(q0, a) = (q0, _, R), (q0, b) = (q1, _, R),
(q1, b) = (q1, _, R), (q1, _) = (q2, _, R)
(a) Specify the execution trace of M on the input string abb.
(b) Provide a regular expression for the language of this Turing
machine.
(c) Suppose we added the following transition to the above machine.
(q1, a) = (q0, _, R)
Provide a regular expression for the language of the resulting Turing
machine.
Assignment 6
2 、 You are required to construct a (deterministic) Turing
machine which takes as input a number n in binary and
subtracts 1 from it. The tape initially contains the
symbol $ followed by the number n in binary. The tape
head is initially scanning the rightmost bit of n. Your
machine should halt with the binary number n−1 on the
tape and the head scanning the rightmost bit again. It is
also required that you replace any leading zeroes by the
$ symbol. We give below the initial and final ID’s for
some sample inputs using the initial state q0 and halting
state qf .
Assignment 6
$1001q01 * $1001qf0
$1000q00 * $$111qf1
You may assume that n ≥ 2, and that the input string is
always in the specified format and does not contain any
leading zeroes.
(a) Give the transitions for this Turing machine and explain
the purpose of each state.
(b) Provide a sample execution trace of your machine for
the input $1010.