Turing Machines

COMP 455 – 002, Spring 2019

Jim Anderson (modified by Nathan Otterness) 1

 Turing Machines
 We want to study computable functions, or algorithms.
 In particular, we will look at algorithms for answering
certain questions.
❖ A question is decidable if and only if an algorithm
exists to answer it.
 Example question: Is the complement of an arbitrary Such an algorithm
CFL also a CFL? does exist for any
❖ This question is undecidable—there is no algorithm specific CFL.
that takes an arbitrary CFL and outputs “yes” if the
complement is a CFL and “no” otherwise.
Jim Anderson (modified by Nathan Otterness) 2
 Undecidability Example
We can give an informal “proof” showing that an
undecidable problem exists. Let’s consider:
 The “Hello World” problem: Given a program 𝑃 and an
input to that program 𝐼, print “yes” if 𝑃 prints “hello
world” when run with input 𝐼 and “no” otherwise.
❖𝑃 can be any program—including one that is very
convoluted!

Jim Anderson (modified by Nathan Otterness) 3

Undecidability Example
Suppose a program, 𝐻, exists that solves the “hello
world” problem:

𝑃 “yes”
𝐻
𝐼 “no”

Now consider another program, 𝐻1 , which is the

same as 𝐻, but prints “hello world” instead of “no”:
𝑃 “yes”
𝐻1
𝐼 “hello world”
Jim Anderson (modified by Nathan Otterness) 4
 Undecidability Example
𝑃 “yes”
𝐻1
𝐼 “hello world”
Next, construct a program 𝐻2 that is identical to 𝐻1
but only takes a single input, 𝑃, that it uses as both 𝑃
and 𝐼 in 𝐻1 :
“yes”
𝑃 𝐻2
“hello world”

Another way to
represent 𝐻2 : 𝑃 “yes”
𝐻1
“hello world”
Jim Anderson (modified by Nathan Otterness) 5
Undecidability Example
Now consider what happens when we run 𝐻2 with
itself as input:
“yes”
𝐻2 𝐻2
“hello world”
We have reached a paradox:
 Suppose 𝐻2 prints “hello world” when fed 𝐻2 as
input. This means that 𝐻2 must also print “yes”
when fed 𝐻2 as input!

Jim Anderson (modified by Nathan Otterness) 6

Turing Machines
 To be able to rigorously do proofs like the “hello
world” example, we need a formal model for defining
computable functions.
 The Turing Machine has become the accepted
model for formalizing functions.

Jim Anderson (modified by Nathan Otterness) 7

 Turing Machines
 A TM can be defined by a 7-tuple 𝑄, Σ, Γ, 𝛿, 𝑞0 , 𝐵, 𝐹
❖ 𝑄: A finite set of states.
A TM’s input starts out on the
❖ Γ: The tape alphabet. “tape”, so the tape alphabet
❖ Σ: The input alphabet. Σ ⊂ Γ. must include the input alphabet.

❖ 𝐵: The blank tape symbol. (𝐵 ∈ Γ, and 𝐵 ∉ Σ)

❖ 𝛿: The next move function: 𝑄 × Γ → 𝑄 × Γ × 𝐿, 𝑅 .
❖ 𝑞0 : The start state. 𝛿 returns a new state,
𝛿 takes a state a new tape symbol,
❖ 𝐹: The set of final states. and tape symbol and whether to move
left or right.
Jim Anderson (modified by Nathan Otterness) 8
Turing Machines
Conceptually, a Turing Machine looks like this:
All of the rest of the tape initially
Non-blank tape symbols is filled with “blank” symbols.
(input symbols, etc.)

… 𝐵 𝐵 𝑋1 … 𝑋𝑖 … 𝑋𝑛 𝐵 𝐵 …
Tape head The “finite control” just
refers to the part of the TM
Finite Control that keeps track of the state.

Initially, the input starts out on the tape, and the tape
head starts at the leftmost input symbol.
Jim Anderson (modified by Nathan Otterness) 9
 Example: Turing Machine Accepting 0𝑛 1𝑛
Input string: 0011
… 𝐵 𝐵 0 0 1 1 𝐵 𝐵 𝐵 …

Finite Control
State = 𝑞0

Initial configuration:
In start state, at
leftmost input symbol
on the tape.

Jim Anderson (modified by Nathan Otterness) 10

 Example: Turing Machine Accepting 0𝑛 1𝑛

0 𝑋
… 𝐵 𝐵 𝑋 0 1𝑌 1𝑌 𝐵 𝐵 𝐵 …

Finite Control
State = 𝑞0 𝑞120 𝑞120 𝑞3 𝑞4

A Turing Machine halts

when it can’t make any
more moves.
Jim Anderson (modified by Nathan Otterness) 11
Example: Formal Notation
𝑄, Σ, Γ, 𝛿, 𝑞0 , 𝐵, 𝐹 , where:
 𝑄 = 𝑞0 , 𝑞1 , 𝑞2 , 𝑞3 , 𝑞4 ,
 Σ = 0, 1 ,
 Γ = 0, 1, 𝑋, 𝑌, 𝐵 ,
 𝐹 = 𝑞4 ,
 And 𝛿 is defined on the following slide…

Jim Anderson (modified by Nathan Otterness) 12

Example: Formal Notation
 𝛿 is as follows:
❖ 𝛿 𝑞0 , 0 = 𝑞1 , 𝑋, 𝑅 : Change the leftmost 0 to X
❖ 𝛿 𝑞0 , 𝑌 = 𝑞3 , 𝑌, 𝑅 : There are no more 0s
❖ 𝛿 𝑞1 , 0 = 𝑞1 , 0, 𝑅 : Skip over 0s
“Seek” to the leftmost 1.
❖ 𝛿 𝑞1 , 𝑌 = 𝑞1 , 𝑌, 𝑅 : Skip over Ys
❖ 𝛿 𝑞1 , 1 = 𝑞1 , 𝑌, 𝐿 : Change the leftmost 1 to Y
❖ 𝛿 𝑞2 , 0 = 𝑞2 , 0, 𝐿 : Skip over 0s
“Seek” to the
❖ 𝛿 𝑞2 , 𝑌 = 𝑞2 , 𝑌, 𝐿 : Skip over Ys
leftmost 0.
❖ 𝛿 𝑞2 , 𝑋 = 𝑞0 , 𝑋, 𝑅 : Move right of the rightmost X
❖ 𝛿 𝑞3 , 𝑌 = 𝑞3 , 𝑌, 𝑅 : Make sure there are no more 1s
❖ 𝛿 𝑞3 , 𝐵 = 𝑞4 , 𝐵, 𝑅 : There were no more 1s, accept.
Jim Anderson (modified by Nathan Otterness) 13
Instantaneous Descriptions for TMs
 As with a PDA, we can also give an instantaneous
description (ID) for a Turing Machine. This indicates both
 An ID for a TM has the following form: 𝛼1 𝑞𝛼2 . the location of the
tape head and the
❖𝑞 corresponds to both the state of the TM and current state (𝑞).
the position of the tape head (𝑞 is written
directly before the tape symbol the head is on).
❖ 𝛼1 𝛼2 = the tape’s current contents, and only We can’t write an
contains the non-blank portion, except in cases infinite number of cells
like 𝛼𝐵𝐵𝐵𝑞 or 𝑞𝐵𝐵𝐵𝛼. without either infinite
input or an infinite
 The depicted portion of the tape is always finite. number of moves.
Jim Anderson (modified by Nathan Otterness) 14
Notation for TM Moves
 Left moves. Suppose 𝛿 𝑞, 𝑋𝑖 = 𝑝, 𝑌, 𝐿 . Then,
❖ 𝑋1 … 𝑋𝑖−1 𝑞𝑋𝑖 … 𝑋𝑛 ├ 𝑋1 … 𝑋𝑖−2 𝑝𝑋𝑖−1 𝑌 … 𝑋𝑛 .
𝑀
❑ The tape head moved from 𝑋𝑖 to 𝑋𝑖−1 .
❑ The state changed from 𝑞 to 𝑝.
❑ 𝑋𝑖 was replaced by 𝑌. We moved left past the start of
❖ Special case where 𝑖 = 1: the tape content, so we need to
show the extra blank symbol (𝐵).
❑ 𝑞𝑋1 𝑋2 … 𝑋𝑛 ├ 𝑝𝐵𝑌𝑋2 … 𝑋𝑛
𝑀
We replaced the rightmost tape
❖ Special case where 𝑖 = 𝑛 and 𝑌 = 𝐵: symbol (𝑋𝑛 ) with a 𝐵, so we no
❑ 𝑋1 … 𝑋𝑛−1 𝑞𝑋𝑛 ├ 𝑋1 … 𝑋𝑛−2 𝑝𝑋𝑛−1 longer include it in the ID.
𝑀
Jim Anderson (modified by Nathan Otterness) 15
Notation for TM Moves
 Right moves. Suppose 𝛿 𝑞, 𝑋𝑖 = 𝑝, 𝑌, 𝑅 . Then,
❖ 𝑋1 … 𝑋𝑖−1 𝑞𝑋𝑖 … 𝑋𝑛 ├ 𝑋1 … 𝑋𝑖−1 𝑌𝑝𝑋𝑖+1 … 𝑋𝑛 .
𝑀
❑ The tape head moved from 𝑋𝑖 to 𝑋𝑖+1 .
❑ The state changed from 𝑞 to 𝑝.
❑ 𝑋𝑖 was replaced by 𝑌. The 𝐵 is included here just
❖ Special case where 𝑖 = 𝑛: to make it clear what the
tape head is pointing at.
❑ 𝑋1 … 𝑋𝑛+1 𝑞𝑋𝑛 ├ 𝑋1 … 𝑋𝑛−1 𝑌𝑝𝐵.
𝑀
❖ Special case where 𝑖 = 1 and 𝑌 = 𝐵:
❑ 𝑞𝑋1 … 𝑋𝑛 ├ 𝑝𝑋2 … 𝑋𝑛 .
𝑀
Jim Anderson (modified by Nathan Otterness) 16
The Language Defined by a TM
The language accepted by a Turing Machine 𝑀 is defined as:
∗
∗
 𝐿 𝑀 ≡ {𝑤 | 𝑤 ∈ Σ and 𝑞0 𝑤 ├ 𝛼1 𝑝𝛼2 for some 𝑝 ∈ 𝐹, and
𝑀
∗
𝛼1 , 𝛼2 ∈ Γ }.
 Assumption: If 𝑀 accepts 𝑤, 𝑀 halts.
 Notation used for moves (similar to a PDA):
𝑖 ∗ 𝑖 ∗
❖ ├, ├, ├, ├, ├.
𝑀 𝑀
❑ The 𝑖 indicates “exactly 𝑖 moves”
❑ The ∗ indicates “any number of moves”
❑ The(optional) 𝑀 below the ├ indicates that TM 𝑀 made the
moves.
Jim Anderson (modified by Nathan Otterness) 17
Recursively Enumerable Languages
 A language that is accepted by a Turing Machine is
called recursively enumerable (RE).
 If a string 𝑤 is in 𝐿 𝑀 , then 𝑀 eventually halts and
accepts 𝑤.
 If 𝑤 ∉ 𝐿 𝑀 , then one of two things can happen:
❖𝑀 halts without accepting 𝑤, or
❖𝑀 never halts.

Jim Anderson (modified by Nathan Otterness) 18

Recursive Languages
 A language that is accepted by a TM that halts on all
inputs is called recursive.
 This is the accepted formal definition of “algorithm”.
❖ For a recursive language, we can always
algorithmically determine membership—but we
can’t necessarily do this for RE languages.
 A problem that is solved by a TM that always halts is
called decidable.
❖ We will discuss decidable problems in much more
detail later!
Jim Anderson (modified by Nathan Otterness) 19
TM Construction Technique: Extra Storage

Storage in the finite control:

 You can keep track of a finite amount of extra data
by incorporating it into the state of a TM.
 Example: Keep track of
an additional symbol: … … …… …
 If the “extra data” can be
𝐴 or 𝐵, and the “state” can
be 𝑞0 or 𝑞1 , then the actual Finite Control
states of the TM can be State = 𝑞0
{[𝑞0 , 𝐴], [𝑞1 , 𝐴], [𝑞0 , 𝐵], Extra data = 𝐴
[𝑞1 , 𝐵]}.
Jim Anderson (modified by Nathan Otterness) 20
Storage in the Finite Control: Example
 Example: Scan the input and ensure the first
symbol doesn’t appear a second time.
 𝑀 = 𝑄, 0, 1 , 0, 1, 𝐵 , 𝛿, 𝑞0 , 𝐵 , 𝐵, 𝐹
 𝑄 = 𝑞0 , 𝑞1 × 0, 1, 𝐵
 𝐹= 𝑞1 , 𝐵
► 𝛿([𝑞0 , 𝐵], 0) = ([𝑞1 , 0], 0, 𝑅) ► 𝛿([𝑞0 , 𝐵], 1) = ([𝑞1 , 1], 1, 𝑅)
► 𝛿([𝑞1 , 0], 1) = ([𝑞1 , 0], 1, 𝑅) ► 𝛿([𝑞1 , 1], 0) = ([𝑞1 , 1], 0, 𝑅)
► 𝛿([𝑞1 , 0], 𝐵) = ([𝑞1 , 𝐵], 0, 𝐿) ► 𝛿([𝑞1 , 1], 𝐵) = ( 𝑞1 , 𝐵 , 0, 𝐿)

 Note: This doesn’t change the fact that the TM has

a finite number of states!
Jim Anderson (modified by Nathan Otterness) 21
Storage in the Finite Control: Example
𝛿 𝑞0 , 𝐵 , 0 = 𝑞1 , 0 , 0, 𝑅
𝛿 𝑞1 , 0 , 1 = 𝑞1 , 0 , 1, 𝑅
If the first symbol
𝛿 𝑞1 , 0 , 𝐵 = 𝑞1 , 𝐵 , 0, 𝐿
was a 0, “store” a 0.
𝛿 𝑞0 , 𝐵 , 1 = 𝑞1 , 1 , 1, 𝑅
𝛿 𝑞1 , 1 , 0 = 𝑞1 , 1 , 0, 𝑅
𝛿 𝑞1 , 1 , 𝐵 = 𝑞1 , 𝐵 , 0, 𝐿

If the first symbol

was a 1, “store” a 1.

This construction may result

in unreachable states.
Jim Anderson (modified by Nathan Otterness) 22
TM Construction Technique: Multiple Tracks

… 𝐵 ¢ 1 0 1 1 $ 𝐵 𝐵 …
… 𝐵 𝐵 𝐵 𝐵 0 1 𝐵 𝐵 𝐵 …
… 𝐵 𝐵 0 0 0 0 0 𝐵 𝐵 …

Finite Control
 In the above figure, the “real” input is between the ¢ and $ on
the first track; the other tracks are used for scratch storage.
 Tape symbols are tuples, i.e., ¢, 𝐵, 𝐵 , 1, 𝐵, 0 , $, 𝐵, 0 , etc.
 This doesn’t change the fact that there are a finite number of
possible tape symbols!
Jim Anderson (modified by Nathan Otterness) 23
Multiple Tracks: Example
 We will use multiple tracks and storage in the
finite control to write a TM that accepts the
language 𝐿 = 𝑤𝑐𝑤 | 𝑤 ∈ 𝐚 + 𝐛 ∗ . Any string 𝑤, followed by a “c”,
 This TM will work by “checking off” followed by a second copy of 𝑤.
corresponding symbols in each copy of 𝑤.

Jim Anderson (modified by Nathan Otterness) 24

 Multiple Tracks: Example
Defining a TM accepting 𝐿 = 𝑤𝑐𝑤 | 𝑤 ∈ 𝐚 + 𝐛 ∗ , using
two tracks and storing an extra symbol in the finite control.
 𝑀 = 𝑄, Σ, Γ, 𝛿, 𝑞0 , 𝐵, 𝐹
 𝑄= 𝑞, 𝑑 | 𝑞 ∈ 𝑞1 , … , 𝑞9 , 𝑑 ∈ 𝑎, 𝑏, 𝐵
 Σ= 𝐵, 𝑑 | 𝑑 ∈ 𝑎, 𝑏, 𝑐
 Γ= 𝑋, 𝑑 | 𝑋 ∈ 𝐵,∗ , 𝑑 ∈ 𝑎, 𝑏, 𝑐, 𝐵
 𝑞0 = 𝑞1 , 𝐵
 𝐹= 𝑞9 , 𝐵
 𝐵 = 𝐵, 𝐵
Jim Anderson (modified by Nathan Otterness) 25
 Multiple Tracks: Example
Defining a TM accepting 𝐿 = 𝑤𝑐𝑤 | 𝑤 ∈ 𝐚 + 𝐛 ∗ , using
two tracks and storing an extra symbol in the finite control.
This TM stores one additional
 𝑀 = 𝑄, Σ, Γ, 𝛿, 𝑞0 , 𝐵, 𝐹 symbol, 𝑑, in its finite control,
along with the “state”.
 𝑄= 𝑞, 𝑑 | 𝑞 ∈ 𝑞1 , … , 𝑞9 , 𝑑 ∈ 𝑎, 𝑏, 𝐵
 Σ= 𝐵, 𝑑 | 𝑑 ∈ 𝑎, 𝑏, 𝑐
 Γ= 𝑋, 𝑑 | 𝑋 ∈ 𝐵,∗ , 𝑑 ∈ 𝑎, 𝑏, 𝑐, 𝐵 The additional symbol that
this TM stores in the finite
 𝑞0 = 𝑞1 , 𝐵 control can be 𝑎, 𝑏, or 𝐵.
This TM has 9
 𝐹= 𝑞9 , 𝐵 logical “states”
 𝐵 = 𝐵, 𝐵
Jim Anderson (modified by Nathan Otterness) 26
 Multiple Tracks: Example
Defining a TM accepting 𝐿 = 𝑤𝑐𝑤 | 𝑤 ∈ 𝐚 + 𝐛 ∗ , using
two tracks and storing an extra symbol in the finite control.
 𝑀 = 𝑄, Σ, Γ, 𝛿, 𝑞0 , 𝐵, 𝐹
The actual input string we care about
 𝑄= 𝑞, 𝑑 | 𝑞 ∈ 𝑞1 , … , 𝑞9 , 𝑑 ∈ 𝑎, 𝑏, 𝐵 consists of 𝑎s, 𝑏s, or 𝑐s. For example,
the symbol 𝐵, 𝑎 corresponds to a
 Σ= 𝐵, 𝑑 | 𝑑 ∈ 𝑎, 𝑏, 𝑐
symbol 𝑎 in an input string.
 Γ= 𝑋, 𝑑 | 𝑋 ∈ 𝐵,∗ , 𝑑 ∈ 𝑎, 𝑏, 𝑐, 𝐵
 𝑞0 = 𝑞1 , 𝐵 The tape has two tracks, so the
 𝐹= 𝑞9 , 𝐵 input “symbols” simply
initialize corresponding cells in
 𝐵 = 𝐵, 𝐵 one of the tracks as blank.
Jim Anderson (modified by Nathan Otterness) 27
 Multiple Tracks: Example
Defining a TM accepting 𝐿 = 𝑤𝑐𝑤 | 𝑤 ∈ 𝐚 + 𝐛 ∗ , using
two tracks and storing an extra symbol in the finite control.
 𝑀 = 𝑄, Σ, Γ, 𝛿, 𝑞0 , 𝐵, 𝐹
 𝑄= 𝑞, 𝑑 | 𝑞 ∈ 𝑞1 , … , 𝑞9 , 𝑑 ∈ 𝑎, 𝑏, 𝐵
 Σ= 𝐵, 𝑑 | 𝑑 ∈ 𝑎, 𝑏, 𝑐 The second track contains
either input symbols or 𝐵.
 Γ= 𝑋, 𝑑 | 𝑋 ∈ 𝐵,∗ , 𝑑 ∈ 𝑎, 𝑏, 𝑐, 𝐵
 𝑞0 = 𝑞1 , 𝐵
 𝐹= 𝑞9 , 𝐵 The first track in the tape contains a ∗ to
indicate if the input symbol in the
 𝐵 = 𝐵, 𝐵 second track has been “checked off”.
Jim Anderson (modified by Nathan Otterness) 28
 Multiple Tracks: Example
To avoid having to write several similar rules,
let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 . 𝛿 =
► 𝛿 𝑞1 , 𝐵 , 𝐵, 𝑑 = 𝑞2 , 𝑑 , ∗, 𝑑 , 𝑅 ► 𝛿 𝑞5 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
► 𝛿 𝑞5 , 𝐵 , ∗, 𝑑 = 𝑞7 , 𝐵 , ∗, 𝑑 , 𝑅
► 𝛿([𝑞2 , 𝑑], 𝐵, 𝑒 ) = ([𝑞2 , 𝑑], 𝐵, 𝑒 , 𝑅)
► 𝛿([𝑞2 , 𝑑], [𝐵, 𝑐]) = ([𝑞3 , 𝑑], [𝐵, 𝑐], 𝑅) ► 𝛿 𝑞6 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
► 𝛿 𝑞6 , 𝐵 , ∗, 𝑑 = 𝑞1 , 𝐵 , ∗, 𝑑 , 𝑅
► 𝛿 𝑞3 , 𝑑 , ∗, 𝑒 = 𝑞3 , 𝑑 , ∗, 𝑒 , 𝑅
► 𝛿 𝑞3 , 𝑑 , 𝐵, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿 ► 𝛿 𝑞7 , 𝐵 , 𝐵, 𝑐 = 𝑞8 , 𝐵 , 𝐵, 𝑐 , 𝑅

► 𝛿 𝑞4 , 𝐵 , ∗, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿 ► 𝛿 𝑞8 , 𝐵 , ∗, 𝑑 = 𝑞8 , 𝐵 , ∗, 𝑑 , 𝑅
► 𝛿 𝑞4 , 𝐵 , 𝐵, 𝑐 = 𝑞5 , 𝐵 , 𝐵, 𝑐 , 𝐿 ► 𝛿 𝑞8 , 𝐵 , 𝐵, 𝐵 = 𝑞9 , 𝐵 , 𝐵, 𝐵 , 𝐿

Jim Anderson (modified by Nathan Otterness) 29

 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)

► 𝛿 𝑞1 , 𝐵 , 𝐵, 𝑑 = 𝑞2 , 𝑑 , ∗, 𝑑 , 𝑅Start off by storing the symbol you’re

►
looking at in the “finite control”
storage, checking off the symbol, and
We’re looking at Store 𝑑 into the starting to move right.
finite-control ❖ The goal will be to make sure the
input symbol 𝑑 in
storage. first “unchecked” symbol in the
the second track.
right-hand string matches the
symbol stored in the finite
control.

Jim Anderson (modified by Nathan Otterness) 30

 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)

► 𝛿 𝑞1 , 𝐵 , 𝐵, 𝑑 = 𝑞2 , 𝑑 , ∗, 𝑑 , 𝑅 ► Start off by storing the symbol you’re

looking at in the “finite control”
storage, checking off the symbol, and
Previously, this starting to move right.
symbol was not So we’ll ❖ The goal will be to make sure the

checked off… check it off. first “unchecked” symbol in the

right-hand string matches the
symbol stored in the finite
control.

Jim Anderson (modified by Nathan Otterness) 31

 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)
The point of state 𝑞2 is just
to move right until we
► 𝛿 𝑞1 , 𝐵 , 𝐵, 𝑑 = 𝑞2 , 𝑑 , ∗, 𝑑 , 𝑅
reach the right-hand string.
► 𝛿([𝑞2 , 𝑑], 𝐵, 𝑒 ) = ([𝑞2 , 𝑑], 𝐵, 𝑒 , 𝑅)
► 𝛿([𝑞2 , 𝑑], [𝐵, 𝑐]) = ([𝑞3 , 𝑑], [𝐵, 𝑐], 𝑅) We know we’ve reached
the right-hand string after
► 𝛿 𝑞3 , 𝑑 , ∗, 𝑒 = 𝑞3 , 𝑑 , ∗, 𝑒 , 𝑅 passing the input symbol 𝑐.
► 𝛿 𝑞3 , 𝑑 , 𝐵, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿

► 𝛿 𝑞4 , 𝐵 , ∗, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿
► 𝛿 𝑞4 , 𝐵 , 𝐵, 𝑐 = 𝑞5 , 𝐵 , 𝐵, 𝑐 , 𝐿

Jim Anderson (modified by Nathan Otterness) 32

 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)

► 𝛿 𝑞1 , 𝐵 , 𝐵, 𝑑 = 𝑞2 , 𝑑 , ∗, 𝑑 , 𝑅
The point of state 𝑞3 is to move
► 𝛿([𝑞2 , 𝑑], 𝐵, 𝑒 ) = ([𝑞2 , 𝑑], 𝐵, 𝑒 , 𝑅)
right until we reach the first
► 𝛿([𝑞2 , 𝑑], [𝐵, 𝑐]) = ([𝑞3 , 𝑑], [𝐵, 𝑐], 𝑅) unchecked symbol in the right-
hand string.
► 𝛿 𝑞3 , 𝑑 , ∗, 𝑒 = 𝑞3 , 𝑑 , ∗, 𝑒 , 𝑅
► 𝛿 𝑞3 , 𝑑 , 𝐵, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿 Once this symbol is reached,
check it off. (The TM will die here
► 𝛿 𝑞4 , 𝐵 , ∗, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿 if the symbol doesn’t match the
► 𝛿 𝑞4 , 𝐵 , 𝐵, 𝑐 = 𝑞5 , 𝐵 , 𝐵, 𝑐 , 𝐿
one in the finite-control storage.)

Jim Anderson (modified by Nathan Otterness) 33

 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)

► 𝛿 𝑞1 , 𝐵 , 𝐵, 𝑑 = 𝑞2 , 𝑑 , ∗, 𝑑 , 𝑅

► 𝛿([𝑞2 , 𝑑], 𝐵, 𝑒 ) = ([𝑞2 , 𝑑], 𝐵, 𝑒 , 𝑅)

► 𝛿([𝑞2 , 𝑑], [𝐵, 𝑐]) = ([𝑞3 , 𝑑], [𝐵, 𝑐], 𝑅)
The point of state 𝑞4 is to
► 𝛿 𝑞3 , 𝑑 , ∗, 𝑒 = 𝑞3 , 𝑑 , ∗, 𝑒 , 𝑅
► 𝛿 𝑞3 , 𝑑 , 𝐵, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿 move left until we reach
the left-hand string again.
► 𝛿 𝑞4 , 𝐵 , ∗, 𝑑 = 𝑞4 , 𝐵 , ∗, 𝑑 , 𝐿
► 𝛿 𝑞4 , 𝐵 , 𝐵, 𝑐 = 𝑞5 , 𝐵 , 𝐵, 𝑐 , 𝐿 Enter state 𝑞5 after
passing the symbol 𝑐.
Jim Anderson (modified by Nathan Otterness) 34
 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)
In 𝑞5 , we are looking at the
last symbol in the left-hand ► 𝛿 𝑞5 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
string. If it’s not checked off, ► 𝛿 𝑞5 , 𝐵 , ∗, 𝑑 = 𝑞7 , 𝐵 , ∗, 𝑑 , 𝑅
go to state 𝑞6 .
► 𝛿 𝑞6 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
If the last symbol in the left- ► 𝛿 𝑞6 , 𝐵 , ∗, 𝑑 = 𝑞1 , 𝐵 , ∗, 𝑑 , 𝑅
hand string was checked off
already, go to 𝑞7 instead. ► 𝛿 𝑞7 , 𝐵 , 𝐵, 𝑐 = 𝑞8 , 𝐵 , 𝐵, 𝑐 , 𝑅

► 𝛿 𝑞8 , 𝐵 , ∗, 𝑑 = 𝑞8 , 𝐵 , ∗, 𝑑 , 𝑅
► 𝛿 𝑞8 , 𝐵 , 𝐵, 𝐵 = 𝑞9 , 𝐵 , 𝐵, 𝐵 , 𝐿

Jim Anderson (modified by Nathan Otterness) 35

 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)
In 𝑞6 , keep moving left
until we find the leftmost ► 𝛿 𝑞5 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
checked-off symbol in the ► 𝛿 𝑞5 , 𝐵 , ∗, 𝑑 = 𝑞7 , 𝐵 , ∗, 𝑑 , 𝑅
left-hand string.
► 𝛿 𝑞6 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
Move right from this ► 𝛿 𝑞6 , 𝐵 , ∗, 𝑑 = 𝑞1 , 𝐵 , ∗, 𝑑 , 𝑅
symbol to get to the next
unchecked symbol in the ► 𝛿 𝑞7 , 𝐵 , 𝐵, 𝑐 = 𝑞8 , 𝐵 , 𝐵, 𝑐 , 𝑅
left-hand string, and repeat
the process from state 𝑞1 . ► 𝛿 𝑞8 , 𝐵 , ∗, 𝑑 = 𝑞8 , 𝐵 , ∗, 𝑑 , 𝑅
► 𝛿 𝑞8 , 𝐵 , 𝐵, 𝐵 = 𝑞9 , 𝐵 , 𝐵, 𝐵 , 𝐿

Jim Anderson (modified by Nathan Otterness) 36

Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)

► 𝛿 𝑞5 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
► 𝛿 𝑞5 , 𝐵 , ∗, 𝑑 = 𝑞7 , 𝐵 , ∗, 𝑑 , 𝑅
In state 𝑞7 , everything in
the left-hand string has ► 𝛿 𝑞6 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
been checked off, so we’re ► 𝛿 𝑞6 , 𝐵 , ∗, 𝑑 = 𝑞1 , 𝐵 , ∗, 𝑑 , 𝑅
getting ready to make sure
everything in the right- ► 𝛿 𝑞7 , 𝐵 , 𝐵, 𝑐 = 𝑞8 , 𝐵 , 𝐵, 𝑐 , 𝑅
hand string is checked off,
too. ► 𝛿 𝑞8 , 𝐵 , ∗, 𝑑 = 𝑞8 , 𝐵 , ∗, 𝑑 , 𝑅
State 𝑞7 ’s main purpose is ► 𝛿 𝑞8 , 𝐵 , 𝐵, 𝐵 = 𝑞9 , 𝐵 , 𝐵, 𝐵 , 𝐿
to move right past the 𝑐.
Jim Anderson (modified by Nathan Otterness) 37
 Multiple Tracks: Example
(Let 𝑑 ∈ 𝑎, 𝑏 and 𝑒 ∈ 𝑎, 𝑏 .)

► 𝛿 𝑞5 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
► 𝛿 𝑞5 , 𝐵 , ∗, 𝑑 = 𝑞7 , 𝐵 , ∗, 𝑑 , 𝑅
In state 𝑞8 , move right while
checking that everything in ► 𝛿 𝑞6 , 𝐵 , 𝐵, 𝑑 = 𝑞6 , 𝐵 , 𝐵, 𝑑 , 𝐿
the right-hand string is ► 𝛿 𝑞6 , 𝐵 , ∗, 𝑑 = 𝑞1 , 𝐵 , ∗, 𝑑 , 𝑅
checked off (if any input
symbol is not checked off, ► 𝛿 𝑞7 , 𝐵 , 𝐵, 𝑐 = 𝑞8 , 𝐵 , 𝐵, 𝑐 , 𝑅
execution will die).
► 𝛿 𝑞8 , 𝐵 , ∗, 𝑑 = 𝑞8 , 𝐵 , ∗, 𝑑 , 𝑅
If we made it past all the input ► 𝛿 𝑞8 , 𝐵 , 𝐵, 𝐵 = 𝑞9 , 𝐵 , 𝐵, 𝐵 , 𝐿
symbols without dying, go to Side note: we already accept here, so it
state 𝑞9 (which is accepting). doesn’t matter whether we move L or R now.
Jim Anderson (modified by Nathan Otterness) 38
 Multiple Tracks: Example

Here’s the TM as a
transition diagram:

Jim Anderson (modified by Nathan Otterness) 39

TM Technique: Shifting Symbols Over

 You may need to create extra space in the middle

of a string of symbols in a TM.
 This can be accomplished by holding a small
buffer of symbols in the finite control.
 Example: Shifting a string of nonblank symbols
over by two spaces.
❖ States will be of the form 𝑞, 𝐴1 , 𝐴2 , where 𝑞 is
𝑞1 or 𝑞2 and 𝐴1 and 𝐴2 are in Γ.

Jim Anderson (modified by Nathan Otterness) 40

 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols
over by two spaces:
► 𝛿 𝑞1 , 𝐵, 𝐵 , 𝐴1 = 𝑞1 , 𝐵, 𝐴1 , 𝑋, 𝑅 , 𝐴1 ∈ Γ − 𝐵, 𝑋
► 𝛿 𝑞1 , 𝐵, 𝐴1 , 𝐴2 = 𝑞1 , 𝐴1 , 𝐴2 , 𝑋, 𝑅 , 𝐴1 and 𝐴2 ∈ Γ − 𝐵, 𝑋

► 𝛿 𝑞1 , 𝐴1 , 𝐴2 , 𝐴3 = 𝑞1 , 𝐴2 , 𝐴3 , 𝐴1 , 𝑅 , 𝐴1 , 𝐴2 , and 𝐴3 ∈ Γ − 𝐵, 𝑋

► 𝛿 𝑞1 , 𝐴1 , 𝐴2 , 𝐵 = 𝑞1 , 𝐴2 , 𝐵 , 𝐴1 , 𝑅 , 𝐴1 and 𝐴2 ∈ Γ − 𝐵, 𝑋
► 𝛿 𝑞1 , 𝐴1 , 𝐵 , 𝐵 = 𝑞2 , 𝐵, 𝐵 , 𝐴1 , 𝐿 , 𝐴1 ∈ Γ − 𝐵, 𝑋

► 𝛿 𝑞2 , 𝐵, 𝐵 , 𝐴 = 𝑞2 , 𝐵, 𝐵 , 𝐴, 𝐿 , 𝐴 ∈ Γ − 𝐵, 𝑋

Jim Anderson (modified by Nathan Otterness) 41

 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols
over by two spaces:
► 𝛿 𝑞1 , 𝐵, 𝐵 , 𝐴1 = 𝑞1 , 𝐵, 𝐴1 , 𝑋, 𝑅 , 𝐴1 ∈ Γ − 𝐵, 𝑋
► 𝛿 𝑞1 , 𝐵, 𝐴1 , 𝐴2 = 𝑞1 , 𝐴1 , 𝐴2 , 𝑋, 𝑅 , 𝐴1 and 𝐴2 ∈ Γ − 𝐵, 𝑋

Start by “shifting” the tape 𝐴1 and 𝐴2 can’t be 𝐵. For simplicity,

symbols 𝐴1 and 𝐴2 into the also assume that they can’t be 𝑋.
finite-control buffer.

We will write the symbol 𝑋 into

the “extra space” we are adding.

Jim Anderson (modified by Nathan Otterness) 42

 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols
over by two spaces:
► 𝛿 𝑞1 , 𝐵, 𝐵 , 𝐴1 = 𝑞1 , 𝐵, 𝐴1 , 𝑋, 𝑅 , 𝐴1 ∈ Γ − 𝐵, 𝑋
► 𝛿 𝑞1 , 𝐵, 𝐴1 , 𝐴2 = 𝑞1 , 𝐴1 , 𝐴2 , 𝑋, 𝑅 , 𝐴1 and 𝐴2 ∈ Γ − 𝐵, 𝑋

► 𝛿 𝑞1 , 𝐴1 , 𝐴2 , 𝐴3 = 𝑞1 , 𝐴2 , 𝐴3 , 𝐴1 , 𝑅 , 𝐴1 , 𝐴2 , and 𝐴3 ∈ Γ − 𝐵, 𝑋

Once the buffer is full, shift subsequent new

symbols into the buffer while replacing them
with the oldest symbol in the buffer.

Jim Anderson (modified by Nathan Otterness) 43

 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols
over by two spaces:
► 𝛿 𝑞1 , 𝐵, 𝐵 , 𝐴1 = 𝑞1 , 𝐵, 𝐴1 , 𝑋, 𝑅 , 𝐴1 ∈ Γ − 𝐵, 𝑋
► 𝛿 𝑞1 , 𝐵, 𝐴1 , 𝐴2 = 𝑞1 , 𝐴1 , 𝐴2 , 𝑋, 𝑅 , 𝐴1 and 𝐴2 ∈ Γ − 𝐵, 𝑋

► 𝛿 𝑞1 , 𝐴1 , 𝐴2 , 𝐴3 = 𝑞1 , 𝐴2 , 𝐴3 , 𝐴1 , 𝑅 , 𝐴1 , 𝐴2 , and 𝐴3 ∈ Γ − 𝐵, 𝑋

► 𝛿 𝑞1 , 𝐴1 , 𝐴2 , 𝐵 = 𝑞1 , 𝐴2 , 𝐵 , 𝐴1 , 𝑅 , 𝐴1 and 𝐴2 ∈ Γ − 𝐵, 𝑋
► 𝛿 𝑞1 , 𝐴1 , 𝐵 , 𝐵 = 𝑞2 , 𝐵, 𝐵 , 𝐴1 , 𝐿 , 𝐴1 ∈ Γ − 𝐵, 𝑋

► 𝛿 𝑞2 , 𝐵, 𝐵 , 𝐴 = 𝑞2 , 𝐵, 𝐵 , 𝐴, 𝐿 , 𝐴 ∈ Γ − 𝐵, 𝑋 Finally, after you encounter blank

symbols, shift the remaining
After shifting everything, state 𝑞2 causes the
symbols out of the buffer.
TM to move left to the newly added space.
Jim Anderson (modified by Nathan Otterness) 44
TM Technique: Shifting Symbols Over

… 𝐵 𝐵 0 0𝑋 1𝑋 1 𝐵 𝐵 𝐵 𝐵 …

𝐴1 , 𝐴2 , 𝐴3 , 𝐴 ∉ 𝑋, 𝐵 :
Finite Control 𝛿 𝑞1 , 𝐵, 𝐵 , 𝐴1 = 𝑞1 , 𝐵, 𝐴1 , 𝑋, 𝑅
State = 𝑞12 𝛿 𝑞1 , 𝐵, 𝐴1 , 𝐴2 = 𝑞1 , 𝐴1 , 𝐴2 , 𝑋, 𝑅
Buffer = 𝐵 , 𝐵 𝛿 𝑞1 , 𝐴1 , 𝐴2 , 𝐴3 = 𝑞1 , 𝐴2 , 𝐴3 , 𝐴1 , 𝑅
𝛿 𝑞1 , 𝐴1 , 𝐴2 , 𝐵 = 𝑞1 , 𝐴2 , 𝐵 , 𝐴1 , 𝑅
𝛿 𝑞1 , 𝐴1 , 𝐵 , 𝐵 = 𝑞2 , 𝐵, 𝐵 , 𝐴1 , 𝐿
Done! 𝛿 𝑞2 , 𝐵, 𝐵 , 𝐴 = 𝑞2 , 𝐵, 𝐵 , 𝐴, 𝐿

Jim Anderson (modified by Nathan Otterness) 45

TM Technique: Subroutines
 TMs can simulate subroutines, even including
parameter-passing and recursion.
 Example: Multiplication using a “copy” subroutine
❖ Given 0𝑚 10𝑛 1 as input, produce 0𝑚𝑛 as output.
❖ This can be done by “copying” 𝑛 0s 𝑚 times.

Jim Anderson (modified by Nathan Otterness) 46

TM Technique: Subroutines
Here’s how the overall TM will work:
1. While processing, the tape will contain a string of the
form 0𝑖 10𝑛 0𝑘𝑛 for some 𝑘.
2. In one “iteration”, we will change a 0 in the first
group of 𝑖 0s to 𝐵 and append 𝑛 0s to the last group of
0s.
❖ This will require copying the group of 𝑛 0s to the
end of the string.
3. Eventually, there will be no more 0s at the start of the
string, and the TM can delete the 10𝑛 1, leaving only
0𝑛𝑚 on the tape.
Jim Anderson (modified by Nathan Otterness) 47
TM Technique: Subroutines
The key component of our “multiplication” TM is this
subroutine, copy, defined in the following diagram:

Jim Anderson (modified by Nathan Otterness) 48

TM Technique: Subroutines
We can “plug in” the copy subroutine where it is
needed in a larger TM:

The “copy”
subroutine goes here.

Jim Anderson (modified by Nathan Otterness) 49

 TM Technique: Subroutines
031q102104
├ 031Xq2010000 …
Example move
├ 031X0q210000 ├ 031XX100000q2
sequence for the
copy routine for ├ 0 31201q 0000
2 ├ 031XX10000q300
an input where … …
𝑚 = 3, 𝑛 = 2, ├ 031X010000q2 ├ 031Xq3X1000000
and 𝑖 = 4: ├ 031X01000q300 ├ 031XXq11000000
├ 031X0100q3000 ├ 031Xq4X1000000
… ├ 031q4X01000000
├ 031q3X0100000 ├ 03q41001000000
├ 031Xq10100000 ├ 031q5001000000
Jim Anderson (modified by Nathan Otterness) 50
 TM Extensions: Multiple Tapes
A multi-tape TM can be
conceptualized like this:
… … ……… … …… …

𝑘 tapes … … ……… … …… …

… … ……… … …… …

Each tape is processed by Finite Control

an independent tape head.
Jim Anderson (modified by Nathan Otterness) 51
TM Extensions: Multiple Tapes
 In a multi-tape TM, each move depends on the state
of the finite control and 𝑘 “current” tape symbols.
 During a move, a multi-tape TM can:
1. Change state.
2. Print a new symbol on each cell scanned by a
tape head.
3. Move each tape head left, right, or stationary.
 We assume the input is initially on the first tape.

Jim Anderson (modified by Nathan Otterness) 52

Single-Tape vs. Multi-Tape TMs
Theorem 8.9: If 𝐿 is accepted by a multi-tape TM,
then it is accepted by a single-tape TM.
Proof:
 Let 𝑀 be a multi-tape TM with 𝑘 tapes.
 We can construct a single-tape TM, 𝑀′ with 2𝑘
tracks that accepts 𝐿 𝑀 .

Jim Anderson (modified by Nathan Otterness) 53

Single-Tape vs. Multi-Tape TMs
𝑀′ ’s (single) multi-track tape looks like this:
Tape 1 of 𝑴  X11 X12  X1i 
Head for tape 1  B B  # 
Tape 2 of 𝑴  X21 X22 X2j  We use the # in these
Head for tape 2  B B #  tracks to indicate where
 the tape heads are.
Tape 𝒌 of 𝑴  Xk1 Xk2  Xkn 
Head for tape 𝒌  B B  # 

The state of 𝑀′ includes:

►The state of 𝑀.
►The number of head markers to the right of the tape head.
►The tape symbols at each head marker.
►The scan direction.
Jim Anderson (modified by Nathan Otterness) 54
 Single-Tape vs. Multi-Tape TMs
Tape 1 of 𝑴  X11 X12  X1i 
Head for tape 1  B B  # 
Tape 2 of 𝑴  X21 X22 X2j 
Head for tape 2  B B # 
Keeping the number of head
 markers to the left of 𝑀′ ’s
Tape 𝒌 of 𝑴  Xk1 Xk2  Xkn  head makes it possible to
Head for tape 𝒌  B B  #  know when all of the head
markers have been visited.
For 𝑀′ to simulate a move of 𝑀:
1. Starting at the leftmost head marker, move right and store the
symbol at each head marker.
2. Determine the move 𝑀 would make.
3. Move left and update the tape symbols and head markers.
4. Change the state as indicated by the move that 𝑀 would make.
Jim Anderson (modified by Nathan Otterness) 55
 Single-Tape vs. Multi-Tape TMs
Tape 1 of 𝑴  X11 X12  X1i 
Head for tape 1  B B  # 
Tape 2 of 𝑴  X21 X22 X2j  Proof (abridged):
Head for tape 2  B B #  After 𝑛 moves in 𝑀, the head
 markers in 𝑀′ can be at most
Xk1 Xk2 2𝑛 cells apart. This means that
Tape 𝒌 of 𝑴   Xkn 
simulating a single move in 𝑀
Head for tape 𝒌  B B  # 
requires 𝑂 𝑛 moves in 𝑀′ .

▶ Theorem 8.10: 𝑀′ takes 𝑂 𝑛2 moves to simulate 𝑛

moves of 𝑀.

▶ Since a multi-tape TM is just as powerful as a single-

tape TM, we can use whichever is more convenient.
Jim Anderson (modified by Nathan Otterness) 56
 TM Extensions: Nondeterministic TMs

 The original definition we discussed was for a

deterministic TM (DTM).
 A nondeterministic TM (NDTM) can “choose” between
multiple possible moves for any combination of state
and tape symbol(s).
 Move function for a nondeterministic TM with 𝑘 tapes:
❖𝛿 maps 𝑄 × Γ 𝑘 to subsets of 𝑄 × Γ × 𝐿, 𝑅, 𝑆 𝑘.

Jim Anderson (modified by Nathan Otterness) 57

TM Extensions: Nondeterministic TMs

Example move function for a nondeterministic

single-tape TM, where Γ = 0, 1, 𝐵 :
 𝛿 𝑞0 , 0 = 𝑞1 , 𝐵, 𝑅 , 𝑞2 , 1, 𝐿

Jim Anderson (modified by Nathan Otterness) 58

Nondeterministic TMs
Note: The book reasons
Theorem 8.11: If 𝐿 is accepted by a single-tape about a multi-tape
NDTM here, instead.
NDTM 𝑀1 , then 𝐿 is accepted by some DTM 𝑀2 .
 Let 𝑑 be the maximum number of nondeterministic
choices 𝑀1 can make at any given move.
 𝑀2 will systematically try all nondeterministic
possibilities.
 𝑀2 will have three tapes.

Jim Anderson (modified by Nathan Otterness) 59

Nondeterministic TMs
How 𝑀2 works:
 Tape 1 holds the input.
 Tape 2 contains a sequence of digits from 1 to 𝑑,
generated systematically. Each sequence dictates a
sequence of choices. e.g.,
1 , 2 , 3 ,…, 𝑑 ,
❖
1,1 , 1, 2 , 2, 1 , … , 𝑑, 𝑑 ,
1,1,1 , 1,1,2 , 1, 2, 1 , … , 𝑑, 𝑑, 𝑑 ,
…
 Tape 3 contains a scratch copy of the input.
Jim Anderson (modified by Nathan Otterness) 60
Nondeterministic TMs
How 𝑀2 works (continued):
1. Generate the next sequence on tape 2
2. Copy tape 1 (input) to tape 3 (scratch copy)
3. Simulate 𝑀1 on tape 3, making choices according
to the sequence on tape 2.
4. If 𝑀1 accepts, then accept;
5. Otherwise, go to step 1.

Jim Anderson (modified by Nathan Otterness) 61

 Nondeterminism and Time Complexity

 Definition: A NDTM 𝑀 is of time complexity 𝑇 𝑛 if

for every accepted string of length 𝑛, some sequence of
at most 𝑇 𝑛 moves leading to an accepting state exists.
Theorem: If 𝐿 is accepted by a single-tape NDTM 𝑀1
with time complexity 𝑇 𝑛 , then 𝐿 is accepted by some
DTM 𝑀2 with time complexity 𝑂 𝑐 𝑇 𝑛 , for some
constant 𝑐.
 Proof: If 𝑀1 in the previous construction can accept in
at most 𝑇 𝑛 moves, then 𝑀2 can accept in at most
𝑂 𝑇 𝑛 𝑑 + 1 𝑇 𝑛 moves.
Jim Anderson (modified by Nathan Otterness) 62
Restricted TMs: Semi-infinite Tape
 In a TM with a semi-infinite tape, the tape is infinite
only to the right of the starting position.
❖ Assume that trying to move left from the
leftmost cell causes execution to die.
Theorem 8.12: (reworded) 𝐿 is recognized by a TM
with a two-way infinite tape if and only if it is
recognized by a TM with a semi-infinite tape.

Jim Anderson (modified by Nathan Otterness) 63

 Restricted TMs: Semi-infinite Tape
Theorem 8.12: 𝐿 is recognized by
a TM with a two-way infinite
Proof of Theorem 8.12: tape if and only if it is recognized
by a TM with a semi-infinite tape.
“If”:
 It’s pretty straightforward to simulate a semi-infinite
tape TM using one with a two-way infinite tape.
❖ Mark the tape cell to the left of the starting
position with a special symbol, and halt without
accepting if that symbol is ever encountered.

Jim Anderson (modified by Nathan Otterness) 64

Restricted TMs: Semi-infinite Tape
Proof (continued), “Only if”:
 Let 𝑀2 = 𝑄2 , Σ2 , Γ2 , 𝛿2 , 𝑞2 , 𝐵, 𝐹2 be a two-way TM.
 Construct a one-way TM 𝑀1 to simulate 𝑀2 .
 The tape of 𝑀1 will have two tracks.

Tape for 𝑀2 : … 𝐴−2 𝐴−1 𝐴0 𝐴1 𝐴2 …

Tape for 𝑀1 : 𝐴0 𝐴1 𝐴2 …
¢ 𝐴−1 𝐴−2 … The ¢ helps us
identify the
Jim Anderson (modified by Nathan Otterness)
leftmost cell. 65
 Restricted TMs: Semi-infinite Tape
Tape for 𝑀2 : … 𝐴−2 𝐴−1 𝐴0 𝐴1 𝐴2 …

Tape for 𝑀1 : 𝐴0 𝐴1 𝐴2 …
¢ 𝐴−1 𝐴−2 …
 Formally, 𝑀1 = 𝑄1 , Σ1 , Γ1 , 𝛿1 , 𝑞1 , 𝐵, 𝐹1 .
❖ 𝑄1 contains all 𝑞, 𝑈 and 𝑞, 𝐷 for all 𝑞 ∈ 𝑄2 .
❑ We’re storing 𝑈 or 𝐷 in the finite control to remember
whether 𝑀1 is on the upper “𝑈”, or lower “𝐷” track.
❑ 𝑄1 also contains 𝑞1 by itself.

❖ Γ1 contains all 𝑋, 𝑌 , where 𝑋 ∈ Γ2 and 𝑌 ∈ Γ2 ∪ ¢ .

Jim Anderson (modified by Nathan Otterness) 66
 Restricted TMs: Semi-infinite Tape
Tape for 𝑀2 : … 𝐴−2 𝐴−1 𝐴0 𝐴1 𝐴2 …

Tape for 𝑀1 : 𝐴0 𝐴1 𝐴2 …
¢ 𝐴−1 𝐴−2 …
 Formally, 𝑀1 = 𝑄1 , Σ1 , Γ1 , 𝛿1 , 𝑞1 , 𝐵, 𝐹1 .
❖ Σ1 contains all 𝑎, 𝐵 , where 𝑎 ∈ Σ2 .
❖ 𝐹1 is 𝑞, 𝑈 , 𝑞, 𝐷 | 𝑞 ∈ 𝐹2 .
❖𝐵 is 𝐵, 𝐵 .

Jim Anderson (modified by Nathan Otterness) 67

 Restricted TMs: Semi-infinite Tape
Tape for 𝑀2 : … 𝐴−2 𝐴−1 𝐴0 𝐴1 𝐴2 …

Tape for 𝑀1 : 𝐴0 𝐴1 𝐴2 …
¢ 𝐴−1 𝐴−2 …
The definition of 𝛿1 :
 𝛿1 𝑞1 , 𝑎, 𝐵 = 𝑞, 𝑈 , 𝑋, ¢ , 𝑅 if 𝛿2 𝑞2 , 𝑎 = 𝑞, 𝑋, 𝑅
❖ This is the case when the first move of 𝑀2 is right.
 𝛿1 𝑞1 , 𝑎, 𝐵 = 𝑞, 𝐷 , 𝑋, ¢ , 𝑅 if 𝛿2 𝑞2 , 𝑎 = 𝑞, 𝑋, 𝐿
❖ This is the case when the first move of 𝑀2 is left.
Jim Anderson (modified by Nathan Otterness) 68
 Restricted TMs: Semi-infinite Tape
Tape for 𝑀2 : … 𝐴−2 𝐴−1 𝐴0 𝐴1 𝐴2 …

Tape for 𝑀1 : 𝐴0 𝐴1 𝐴2 …
¢ 𝐴−1 𝐴−2 …
The definition of 𝛿1 (continued):
For all 𝑋, 𝑌 ∈ Γ1 , with 𝑌 ≠ ¢, and 𝐴 = 𝐿 or 𝑅: If we’re in the lower track
in 𝑀1 we need to move in
 𝛿1 𝑞, 𝑈 , 𝑋, 𝑌 = 𝑝, 𝑈 , 𝑍, 𝑌 , 𝐴 if 𝛿2 𝑞, 𝑋 = 𝑝, 𝑍, 𝐴 . the opposite direction from
❖ This simulates 𝑀2 on the upper track. how 𝑀2 would move.
 𝛿1 𝑞, 𝐷 , 𝑋, 𝑌 = 𝑝, 𝐷 , 𝑋, 𝑍 , 𝐴 if 𝛿2 𝑞, 𝑌 = 𝑝, 𝑍, 𝐴ҧ
❖ This simulates 𝑀2 on the lower track.
Jim Anderson (modified by Nathan Otterness) 69
 Restricted TMs: Semi-infinite Tape
Tape for 𝑀2 : … 𝐴−2 𝐴−1 𝐴0 𝐴1 𝐴2 …

Tape for 𝑀1 : 𝐴0 𝐴1 𝐴2 …
¢ 𝐴−1 𝐴−2 …
The definition of 𝛿1 (continued):
 𝛿1 𝑞, 𝑈 , 𝑋, ¢ = 𝛿1 𝑞, 𝐷 , 𝑋, ¢ = 𝑝, 𝐶 , 𝑌, ¢ , 𝑅 ,
if 𝛿2 𝑞, 𝑋 = 𝑝, 𝑌, 𝐴 , where
𝑈 if 𝐴 = 𝑅
𝐶=ቊ
𝐷 if 𝐴 = 𝐿
 This allows 𝑀1 to switch tracks if it is currently at the leftmost cell.
Jim Anderson (modified by Nathan Otterness) 70
 Restricted TMs: Two-Stack Machine
 Imagine a PDA that can make decisions and modify two
stacks rather than one. This is called a two-stack machine.
Theorem 8.13: A two-stack machine can simulate a TM.
Proof sketch:
TM tape content: … 𝐴1 𝐴2 𝐴3 𝐴4 𝐴5 𝐴6 …

Tape head

𝐴3 𝐴4
Store tape contents in
two stacks like this: 𝐴2 𝐴5 See Section 8.5.2 of the book for
𝐴1 𝐴6 a much more detailed proof.
Jim Anderson (modified by Nathan Otterness) 71
Restricted TMs: Counter Machines
 A counter machine can be thought of as a multi-stack PDA,
with only two stack symbols and some restrictions:
❖ 𝑍0 , serving as the “bottom of stack marker”
❖𝑋
❖ Initially,
only 𝑍0 is on each stack.
❖ Only 𝑋’s can be pushed or popped from the stack.
 Essentially, each stack is a counter. The PDA can only do
three things with each stack:
❖ Increase the “counter” by pushing more 𝑋’s.
❖ Decrement the “counter” by popping a single 𝑋.
❖ Check if the “counter” is 0 by seeing if 𝑍0 is on top of the
stack.
Jim Anderson (modified by Nathan Otterness) 72
Restricted TMs: Three-Counter Machines

Theorem 8.14: Every RE language is accepted by a

three-counter machine.
Proof:
We will show that one stack can be simulated by two
counters, the second of which is a “scratch” counter
used for calculations.
When simulating two stacks, we can then use the
same scratch counter, so three counters in total are
sufficient.

Jim Anderson (modified by Nathan Otterness) 73

 Restricted TMs: Three-Counter Machines
We now need to show how to simulate one stack using
two counters.
 Suppose the stack alphabet contains 𝑟 − 1 symbols.
 We can assume these symbols are denoted 1, … , 𝑟 − 1.
 Store the stack contents 𝑋1 𝑋2 … 𝑋𝑛 (where 𝑋1 is the top
of the stack) as a base-𝑟 number:
𝑋𝑛 𝑟 𝑛−1 + 𝑋𝑛−1 𝑟 𝑛−2 + … + 𝑋2 𝑟 + 𝑋1 .
 Example: Assume 𝑟 = 10 and the stack contains the
symbols 9, 2, 5, and 3 (9 is on top). The counter value
associated with this stack is simply 3259.
Jim Anderson (modified by Nathan Otterness) 74
Restricted TMs: Three-Counter Machines

We now need to show how to simulate stack

operations using a counter.
 Pop the stack: Replace the counter value 𝑖 by 𝑖/𝑟.
The remainder is the old top-of-stack symbol,
which can be stored in the finite control.
3259 3259
❖ Example: popping 3259. = = 325, with
𝑟 10
remainder 9.

Jim Anderson (modified by Nathan Otterness) 75

Restricted TMs: Three-Counter Machines

 Push 𝑋 onto the stack: Replace the counter value 𝑖

by 𝑖𝑟 + 𝑋.
❖ Example: pushing a 6 onto 3259. 3259𝑟 + 6 =
3259 ∗ 10 + 6 = 32596.
 Replace 𝑋 by 𝑌 on top of the stack: If 𝑌 > 𝑋, then
increment 𝑖 by 𝑌 − 𝑋. If 𝑌 < 𝑋, then decrement it
by 𝑋 − 𝑌.

Note that these operations (including pop) only

involve constant values (based on the constant 𝑟).
Jim Anderson (modified by Nathan Otterness) 76
Restricted TMs: Three-Counter Machines

But how do we multiply or divide when we can only

check if a counter is 0 or nonzero?
To multiply a counter by 𝑟 using a scratch counter:
scratch := 0;
while count ≠ 0:
count := count – 1; This algorithm only adds
scratch := scratch + r; or subtracts constant
/* If count was initially 𝑖, then scratch is now 𝑖𝑟. values, and only needs to
So now, copy scratch back to count. */
check if counters are 0.
while scratch ≠ 0:
count := count + 1;
scratch := scratch – 1;
Jim Anderson (modified by Nathan Otterness) 77
Restricted TMs: Three-Counter Machines
To divide a counter by 𝑟 using a scratch counter and
a bounded value 𝑘 (𝑘 ≤ 𝑟, and 𝑟 is constant, so we
can keep track of 𝑘 using a finite number of states):
scratch := 0;
while count ≠ 0:
k := 0;
while k ≠ r and count ≠ 0:
count := count – 1;
k := k + 1;
if k = r:
scratch := scratch + 1;
/* If count was initially 𝑖, then scratch is now 𝑖/𝑟 and k is the remainder.*/
while scratch ≠ 0:
count := count + 1;
scratch := scratch – 1;
Jim Anderson (modified by Nathan Otterness) 78
 Restricted TMs: Two-Counter Machines

We have now shown that we can simulate a single

stack using two counters, and two stacks using three
counters, but we can go farther!
Theorem 8.15: Every RE language is accepted by a two-
counter machine.

Proof: We will show that we can simulate three counters

using two counters.
If 𝑖, 𝑗, and 𝑘 are the three counts we want to track, we
can store them in a single integer 𝑛 = 2𝑖 3𝑗 5𝑘 .
Jim Anderson (modified by Nathan Otterness) 79
 Restricted TMs: Two-Counter Machines
(From previous slide) If 𝑖, 𝑗, and 𝑘 are the three counts we want
to track, we can store them in a single integer 𝑛 = 2𝑖 3𝑗 5𝑘 .
 To increment 𝑖: Replace 𝑛 by 2𝑛.
 To decrement 𝑖: Replace 𝑛 by 𝑛/2.
 To test if 𝑖 is 0: Check if 𝑛 is not divisible by 2.
 The other counters are similar (e.g. increment 𝑗 by replacing 𝑛
with 3𝑛 and increment 𝑘 by setting 𝑛 = 5𝑛, etc.).
We have already seen how to multiply and divide by constant
values using a second “scratch” counter.
Note that this construction works because 2, 3, and 5 are
relatively prime.
Jim Anderson (modified by Nathan Otterness) 80
Restricted TMs: Summary
 We have shown that the following machines are
equally powerful in terms of languages they can identify:
❖ Nondeterministic TMs
❖ Multi-tape TMs
❖ Multi-track TMs
❖ Single-tape TMs
❖ Semi-infinite tape TMs
❖ Two-stack machines
❖ Three-counter machines
❖ Two-counter machines
Jim Anderson (modified by Nathan Otterness) 81
TMs vs. “Real” Computers
We want to show that anything a TM can do is possible using
a computer, and vice versa.
 Designing a computer that simulates a TM is easy: just
store the transition function in a table, which tells a
program what to do next.
 One “gotcha”: a TM has infinite memory.
❖ The book says we can just continually add new disks to
a TM when old ones are full.
❖ What if we have TM that uses more tape cells than
atoms in the universe? (Clearly the book’s approach
isn’t a true Turing Machine.)
❖ My take: we can’t simulate infinite memory, but we will
also never have a practical application that requires it.
Jim Anderson (modified by Nathan Otterness) 82
 Simulating a Computer using a TM
We will consider a simplified model of a computer
called a random access machine (RAM):
A potentially infinite number
of values stored in memory: A finite number of registers:

𝑣1 Both memory and Register 1

registers can store any
𝑣2 integer; even Register 2
𝑣3 indefinitely large ones.
…
𝑣4 Program Counter
…
Program “code” and data
is stored in memory.
Jim Anderson (modified by Nathan Otterness) 83
Random Access Machines
 In a RAM, a “program” is a list of instructions.
❖ Instructions are like what you’d find on a “real”
computer, and consist of an opcode and an
address.
❖ Instructions are encoded as a single integer (e.g.
with some bits representing the opcode and the
remaining bits representing the argument).
 With this model, you can have basic instructions
like LOAD, STORE, ADD, etc., like an assembly
language.

Jim Anderson (modified by Nathan Otterness) 84

Simulating a Random Access Machine

Theorem: A TM can simulate a RAM, provided that

the TM can simulate each individual RAM instruction.
 For example, you can’t have a RAM instruction that
solves the halting problem.
Proof sketch:
We will use multiple tapes to store the RAM’s memory
and register values in a TM.

Jim Anderson (modified by Nathan Otterness) 85

Simulating a Random Access Machine

 One tape in our TM holds memory values that

have been written.
❖ The format of this tape is #0*𝑣0 #1*𝑣1 #10*𝑣2 #...
❖ The # and * are simply delimiters around the
“address” of each value. Assume that each 𝑣𝑖 is
stored in binary.
 Use one additional tape for each register.

Jim Anderson (modified by Nathan Otterness) 86

 Simulating a Random Access Machine

Overview of the TM’s behavior for simulating the RAM:

1. Read the current PC value 𝑖 from the tape holding the
“program counter” register.
2. Scan the memory tape for the “address” #𝑖*. Halt if it
isn’t found.
3. Decode the instruction opcode and address from the
memory value at location 𝑖.
4. Perform the operation indicated by the opcode and
address.
5. Repeat from step 1.
Jim Anderson (modified by Nathan Otterness) 87
 Simulating a Random Access Machine
Possible behavior for a couple common “instructions”:
 If the instruction indicates to “ADD” a value at address
𝑗 to a register:
1. Scan memory for #𝑗*, and add the value found
afterwards to a register.
2. Add 1 to the program counter register to advance to
the next instruction.
 If the instruction indicates to “GOTO” address 𝑗:
1. Copy the address 𝑗 to the program counter register.
(Other instructions can be simulated in a similar manner)
Jim Anderson (modified by Nathan Otterness) 88
Running Time of the RAM Simulation

The previous simulation will run in polynomial time

provided that a couple restrictions hold:
 First, the RAM can’t have an instruction that takes
exponential time to compute.
❖ For example, you can’t have a “solve the
traveling salesman problem” instruction.
❖ This isn’t a big restriction, because no real
computer will have instructions like this anyway.

Jim Anderson (modified by Nathan Otterness) 89

 Running Time of the RAM Simulation

(Continued list of restrictions for this simulation)

 Second, we can’t have instructions that “drastically” increase
the length of some number.
❖ Example: If integer values are unrestricted, then starting
with a value 2, applying a hypothetical “multiply a value
2𝑛
by itself” instruction 𝑛 times would lead to a value of 2 ,
which takes 2𝑛+1 bits to represent. Just writing these
down takes exponential time, so we don’t allow
instructions like this.
❖ In reality, this is also a mild restriction. In real computers,
the lengths of individual numbers are limited.

Jim Anderson (modified by Nathan Otterness) 90

Running Time of the RAM Simulation
Theorem 8.17: If a computer has (1) only instructions that
increase the length of a number by 1 bit and (2) has only
instructions that a multi-tape TM can perform on
numbers of length 𝑘 in 𝑂 𝑘 2 steps, then a TM exists that
can simulate 𝑛 steps of the computer in 𝑂 𝑛3 moves.
Notes:
 (1) rules out multiplication, which can double the
length of a number. However, multiplication can be
simulated in polynomial time by repeatedly adding,
which is allowed.
 The 𝑂 𝑘 2 bound in (2) was selected because it seems
sufficient for most “reasonable” instructions.
Jim Anderson (modified by Nathan Otterness) 91
Running Time of the RAM Simulation

We won’t prove Theorem 8.17 in detail. However,

 If individual instructions can be simulated in
polynomial time, we only need to worry about
spending an exponential amount of time scanning
the “memory” tape.
 Initially, the “memory” tape holds the program to
execute and input data, all of which starts at a
“constant” length.
 Restriction (1) ensures that the tape’s length
remains polynomial in 𝑛 (after executing 𝑛 RAM
instructions). See book Section 8.6.3 for the proof.
Jim Anderson (modified by Nathan Otterness) 92
Final Comment on RAM Simulation
 We know from before that a single-tape TM can
simulate a multi-tape TM in polynomial time.
 So, even a single-tape TM can simulate a RAM
computation in polynomial time.

Jim Anderson (modified by Nathan Otterness) 93