CS 208: Computing Theory

Assoc. Prof. Dr. Brahim Hnich

Faculty of Computer Sciences
Izmir University of Economics
Computability Theory

Turing Machines
 TMs

 We introduce the most powerful of the automata

we will study: Turing Machines (TMs)

 TMs can compute any function normally

considered computable

 Indeed we can define computable to mean

computable by a TM
 Turing Machines
 A Turing machine (TM) can compute everything a
usual computer program can compute
 may be less efficient

 But the computation allows a mathematical

analysis of questions like :
 What is computable?
 What is decidable?
 What is complexity?
 input
State
a a b a - - -
control

A Turing Machine is a theoretical computer consisting of a tape

of infinite length and a read-write head which can move left and
right across the tape.
 Informal definition

 TMs uses an infinite memory (tape)

 Tape initially contain input string followed by
blanks
 When started, a TM executes a series of discrete
transitions, as determined by its transition
function and by the initial characters on the tape
 Informal definition
 For each transition, the machine checks
 what state it is in and what character is written on the
tape below the head.
 Based on those, it then changes to a new state, writes
a new character on the tape, and moves the head one
space left or right.
 The machine stops after transferring to the
special HALT (accept or reject) state.
 If TM doesn’t enter a HALT state, it will go on
forever
 Differences between FA and TMs

 A TM can both write on the tape and read from it

 The read-write head can move both to the left and
to the right
 The tape is infinite
 The special states of rejecting and accepting take
immediate effect
 Examples

aa,R
ba,R

aa,R means TM reads the symbol a, it replaces it with a

and moves head to right

This TM when reading an a will move right one square stays in the
same start state. When it scans a b, it will change this symbol to an
a and go into the other state (accept state).
 Examples

 A TM that tests for memberships in the language

 A= {w#w | w belongs to {0,1}*}
 Idea:
 Zig-zag across tape, crossing off matching symbols
 Examples

a b b # a b b X b b # a b b X
b b b # a b b

 Tape head starts over leftmost symbol

 Record symbol in control and overwrite X
 Scan right: reject if blank encountered before #
 When # encountered, move right one space
 If symbols don’t match, reject
 Examples

X
b b b # X b b X X b # X b b X
b X b # X b b

 Overwrite X
 Scan left, past # to X
 Move one space right
 Record symbol and overwrite X
 Move right to #, when X encountered, move right one space
 If symbols don’t match, reject
 …
 Examples

X
b X X # X X X

 Finally scan left

 If a or b encountered, reject
 When blank encountered, accept
 Formal Definition

 A Turing Machine is a 7-tuple (Q,∑,T, ξ,q0,qaccept,

qreject), where
 Q is a finite set of states
 ∑ is a finite set of symbols called the alphabet
 T is the tape alphabet, where – belongs to T, and ∑⊆ T
 ξ : Q x T  Q x T x {L,R} is a partial function
 q0 Є Q is the start state
 qaccept ⊆ Q is the accept state
 qreject ⊆ Q is the reject state, where qaccept ≠ qreject
 Transition function
 ξ : Q x T  Q x T x {L,R} is the transition function
ξ(q,a) = (r,b,L) means

in state q where head reads tape symbol a

the machine overwrites a with b

enters state r

move the head left

 Workings of a TM

 M = (Q,∑,T, ξ,q0,qaccept, qreject) computes as follows

 Input w=w1w2…wn is on leftmost n tape sqaures

 Rest of tape is blank –

 Head is on leftmost square of tape

 Workings of a TM

 M = (Q,∑,T, ξ,q0,qaccept, qreject)

 When computation starts
 M Proceeds according to transition function ξ

 If M tries to move head beyond left-hand-end of tape, it

doesn’t move

 Computation continues until qaccept or qreject is reached

 Otherwise M runs forever

 Configurations

 Computation changes
 Current state
 Current head position
 Tape contents
 Configurations

 Configuration
 1011r0111
 Means
 State is r
 Left-Hand-Side (LHS) is 1011
 Right-Hand-Side (RHS) is 0111
 Head is on RHS 0
 Configurations
 uarbv yields upacv if
 ξ(r,b) = (p,c,L)

 uarbv yields uacpv if

 ξ(r,b) = (p,c,R)

 Special cases: rbv yields pcv if

 ξ(r,b) = (p,c,L)

 wr is the same as wr--

 More configurations
 We have
 starting configuration q0w

 accepting configuration w0qacceptw1

 rejecting configuration w0qrejectw1

 halting configurations
 w0qacceptw1
 w0qrejectw1
 Accepting a language

 TM M accepts input w if a sequence of

configurations C1,C2,…,Ck exist
 C1 is start configuration of M on w
 Each Ci yields Ci+1
 Ck is an accepting configuration
 The collection of strings that M accepts is the
language of M, denoted by L(M)
 Example: TM that checks its third symbol is 0, accepts
if so, and runs forever, if not (note it never rejects).
Example: : { anbm | n,m ≥ 0} .
Example: : { anbncn | n ≥ 0} .
 An Example

M accepting {w#w |w in {0,1}*}

String matching problem ?
 Detailed example(M1 accepting {w#w |w in {0,1}*})
TM for L(M1)
 An Example

M accepting 02^n
What is the language ?
 Detailed example(M1 accepting 02^n)
TM for L(M1): Cross off every other zero
Adding two to a number in unary
Adding 1 to a Binary Number
Shift right machine
Shift right machine
 Detailed Example

 L = {aibjck | ixj=k and i,j,k>0}

 L = {anbncn | n>0}

 L = {#x1#x2# … #xn | each xi in {0,1}* and xi!=xj for

each i!=j}
 Details
There are 3 level of details we might use to describe TM’s
Give the complete formal description
 the states, alphabet, transition, etc.
Implementation description
 Partition the states into stages, Use English to describe how each stage move the head
and stores data on the tape
 Each stage performs one function.
 Describe how we move between stages
 No details of states or transition function
High level description
 Use English to describe an algorithm
 Don’t mention how to describe tape or moves
 Enumerable languages

 Definition: A language is (recursively) enumerable

if some TM “recognizes” it
 In some other textbooks Turing-recognizable
 Enumerable languages

 On an input to a TM we may
 accept
 reject
 loop (run for ever)
 Not very practical: never know if TM will halt
 Enumerable languages
 Definition: A TM decides a language if it always
halts in an accept or reject state (that is, it never
loops). Such a TM is called a decider
 Definition: A language is decidable if some TM
decides it
 Some textbooks use recursive instead of decidable
 Recursive Language: decided by an always-halting TM
 Therefore, every decidable (or recursive) language
is (recursively) enumerable, but not the reverse!
 NOTE: Recursively Enumerable (or Turing recognizable) means Turing machine does not necessarily halt. Recursive
(or decidable), on the other hand, means that the Turing machine always halts.
 Decidable vs recognizable
 Decidable (always halting TM)
 Recursive
 Computable
 Solvable
 Recognizable (recognize members only)
 Recursively enumerable (RE)
 Partially decidable
 Semi decidable
 Unrecognizable (can’t even recognize members reliably)
 Not RE
 Not partially decidable
 Example of decidable language

 Here is a decidable language

 L={aibjck | ix j = k, i,j,k > 0}
 Because there exist a TM that decides it
 How?
 Example of decidable language
 How?
1. Scan from left to right to check that input is of the form a*b*c*
2. Return to the start
3. Cross off an a and
4. scan right till you see a b
5. Mark each b and scan right till you see a c
6. Cross off that c
7. Move left to the next b, and repeat previous two steps until all b’s
are marked
8. Unmark all b’s and go to the start of the tape
9. Goto step 1 until all a’s are crossed off
10. Check if all c’s are crossed off; if so accept; otherwise reject
 Example of decidable language

 given,
 aabbbcccccc
 Example of decidable language

 given,
 aabbbcccccc
 aabbbcccccc
 abbbcccccc

 Example of decidable language

 given,
 aabbbcccccc
 aabbbcccccc
 abbbcccccc
 abbbcccccc

 Example of decidable language

 given,
 aabbbcccccc
 aabbbcccccc
 abbbcccccc
 abbbcccccc
 abbbccccc

 Example of decidable language

 given,
 aabbbcccccc
 aabbbcccccc
 abbbcccccc
 abbbcccccc
 abbbccccc
 abbbcccc
 abbbccc

 Example of decidable language
 given,
 aabbbcccccc
 aabbbcccccc
 abbbcccccc
 abbbcccccc
 abbbccccc
 abbbcccc
 abbbccc
 abbbccc
 bbbccc bbbcc bbbc bbb
 TM: Variants

 What about a TM that has the ability to move

Right, move Left, and “Stay” put.
 We can easily model this modified TM with basic TM.
 Multitape Turing Machine
 Does not add any power
 Nondetermisim added to TMs
 Does not add any power
 Simulating a Multitape TM

 S simulates M
 Simulating a Nondeterministic TM
1) Depth first search may continue forever, hence breadth first search.
2) Tape 1 contains the input and is never altered.
3) Tape 2 maintains a copy of N’s tape on some branch.
4) Tape 3 tracks D’s location in N’s nondeterministic computation tree.
 Remarks

 Many models have been proposed for general-

purpose computation
 Remarkably all “reasonable” models are
equivalent to Turing Machines
 All ”reasonable” programming languages like C,
C++, Java, Prolog, etc are equivalent
 The notion of an algorithm is model-independent!
 Equivalent models
 Evidence. Many, many models of computation have turned out to be equivalent (universal).
 models of computation description
 enhanced Turing machines multiple heads, multiple tapes, 2D tape, nondeterminism
 untyped lambda calculus method to define and manipulate functions
 recursive functions functions dealing with computation on integers
 unrestricted grammars iterative string replacement rules used by linguists
 extended Lindenmayer systems parallel string replacement rules that model plant growth
 programming languages Java, C, C++, Perl, Python, PHP, Lisp,
PostScript, Excel
 random access machines registers plus main memory, e.g., TOY, Pentium
 cellular automata cells which change state based on local interactions
 quantum computer compute using superposition of quantum states
 DNA computer compute using biological operations on DNA
 PCP systems string matching puzzles (stay tuned)
 What is an algorithm?

 Informally
 A recipe
 A procedure
 A computer program
 Historically
 Notion has long history in Mathematics (Al-khwarzmi),
but
 Not precisely defined until 20th century
 Informal notion rarely questioned, but are insufficient
 Hilbert’s 10th problem

 Hilbert’s tenth problem (1900): Find a finite

algorithm that decides whether a polynomial has
an integer root.
 What is a polynomial?
 A polynomial is a sum of terms, each term is a product of
variables and constants (coefficients)
 A root of a polynomial is an assignment of values to the
variables such the value of the polynomial is 0
 x=2 and y=2 is a root for 5x +15y=40
 There is no such algorithm (1970)!
 Mathematics of 1900 could not have proved this, because they
didn’t have a formal notion of an algorithm
 Formal notions are required to show that no algorithm exists
 Church-Turing Thesis
 Formal notions of ‘computable’ appeared in 1936
 Lambda-calculus of Alonzo Church
 Turing machines of Alan Turing
 These definitions look very different, but are equivalent
 The Church-Turing Thesis:
 The intuitive notion of algorithms equals Turing machines
algorithms
 Church's Thesis is a belief, not a theorem. Though we often act as
if we believe it is true, even though we don’t know it is true.
 In 1970, it was shown that no algorithm exists for testing whether a
polynomial has integral roots
 Algorithmically computable means computable by a TM
Relationships among machines

NPDA

NFA/DFA TM

Linear Bounded Nondeterministic TM

Relationships among languages

Context-free

regular Recursively
enumerable

Context Sensitive
 Chomsky hierarchy
 Type-0 grammars (unrestricted grammars) include all formal grammars. They generate
exactly all languages that can be recognized by a Turing machine, called recursively
enumerable languages.
 Type-1 grammars (context-sensitive grammars) generate the context-sensitive
languages. The languages described by these grammars are exactly all languages that
can be recognized by a linear bounded automaton (a nondeterministic Turing machine
whose tape is bounded by a constant times the length of the input.)
 Type-2 grammars (context-free grammars) generate the context-free languages. These
languages are exactly all languages that can be recognized by a non-
deterministic pushdown automaton. Context-free languages are the theoretical basis for
the syntax of most programming languages.
 Type-3 grammars (regular grammars) generate the regular languages. Regular
languages are commonly used to define search patterns and the lexical structure of
programming languages.
 Note that the set of grammars corresponding to recursive languages is not a member of
this hierarchy.
 Closures Of Languages
Language/Operation Union Intersection Complement Concatenation Kleen Reversal Intersection
e Star with Regular

Regular(Type 3) Yes Yes Yes Yes Yes Yes Yes

Deterministic Context Free No No Yes No No No Yes

Context Free(Type 2) Yes No No Yes Yes Yes Yes

Context Sensitive(Type 1) Yes Yes Yes Yes Yes Yes Yes

Recursive(Turing Decidable) Yes Yes Yes Yes Yes Yes Yes

Recursive Enumerable(Turing Yes Yes No Yes Yes Yes Yes

Recognizable)(Type 0)
 Problem

 How to recognize left end of the tape ?

 Put a special symbol $ on the left and shift the entire
input over 1 cell to the right
 Design the TM for that ????
 Build a TM to ‘recognize’ the language 0 n1n0n.
Build a TM to ‘decide’ the language 0n1n0n.
Use the TM for 0n1n as subroutine.
 Since 0n1n0n is not CFL, so
 CFL ⊂ Decidable Languages
 Conclusions
Turing Machines
definition
examples

Enumerable/decidable
languages
definition

Next: Decidability Theory